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Abstract 

Congress  has  mandated  the  Air  Foree  to  reform  and  streamline  its  aequisition 
procedures  while  promoting  participation  by  small  business  contractors.  Reports  on  the 
federal  government  as  a  whole  suggest  that  the  acquisition  reforms  have  had  a  negative 
effect  on  small  business  participation,  with  a  declining  number  of  small  businesses 
receiving  contract  awards. 

Contracting  with  small  business  should  be  done  for  reasons  beyond  Congressional 
mandates.  Small  businesses’  innovativeness  and  entrepreneurialism  provide  benefits  that 
the  Air  Force  should  seek  to  maximize. 

A  causal  regression  model  is  used  to  determine  that  the  Air  Force  has  actually 
increased  the  number  of  small  businesses  receiving  contract  awards  over  the  past  thirteen 
years  and  has  maintained  a  steady  stream  of  new  small  businesses  into  that  group. 
Variables  are  tested  to  determine  their  effect  on  small  business  participation.  Those  tests 
reveal  that  number  of  contract  awards  and  number  of  government-wide  agency  contract 
orders  placed  are  useful  for  predictive  purposes. 

The  research  also  identifies  two  performance  measurements  for  the  Air  Force 
small  business  office:  total  number  of  small  business  contractors  and  total  number  of  new 
small  business  contractors.  Goals  for  these  metrics  and  strategies  to  reach  those  goals 
should  be  developed  by  the  small  business  program. 
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MEASURING  SMALL  BUSINESS  PARTICIPATION  IN  AIR  LORCE 


CONTRACTING;  THE  IMPACT  OP  ACQUISITION  REPORM 


I.  Introduction 


Background 

As  President  George  W.  Bush  remarked  in  St.  Louis,  Missouri  on  January  22,  2003 
“It’s  important  for  our  fellow  Amerieans  to  understand  that  the  strength  of  our  eountry, 
the  strength  of  our  eeonomy  really  depends  upon  the  strength  of  the  small  business 
eommunity  all  aeross  Ameriea”  (Bush,  2003).  Small  businesses  are  an  integral  part  of 
the  Ameriean  eeonomy.  Reasons  why  small  businesses  are  vital  are  refleeted  in  President 
Bush’s  Small  Business  Agenda.  “Small  businesses  are  the  heart  of  the  Ameriean 
eeonomy  beeause  they  drive  innovation  .  .  .  these  dynamie  eompanies  also  drive  the  job 
ereation  proeess  .  .  .  entrepreneurship  has  beeome  the  path  to  prosperity”  (The  President’s 
Small  Business  Agenda,  2002:1). 

Reeognizing  the  innovativeness  and  entrepreneurial  spirit  that  small  businesses 
eontribute  to  the  eeonomy.  Congress  and  the  president  have  enaeted  legislation  over  the 
past  fifty  years  to  help  proteet  and  develop  small  businesses.  Mueh  of  this  legislation 
foeuses  on  ensuring  fair  eompetition  in  federal  government  eontraeting  for  small 
businesses.  Prom  the  Small  Business  Aet  of  1953  to  bills  pending  in  the  eurrent 
Congress,  the  leadership  of  the  federal  government  has  demonstrated  a  eommitment  to 
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small  businesses.  It  is  this  eommitment  that  led  to  the  development  of  small  business 
programs  in  every  federal  ageney  and  established  speeifie  goals  of  pereentages  of 
eontracts  awarded  to  small  businesses  for  those  ageneies. 

But  despite  this  eommitment,  problems  exist  in  federal  government  eontraeting 
efforts  with  small  business.  Congresswoman  Nydia  M.  Velazquez  serves  as  the  ranking 
Demoeratie  member  of  the  House  Small  Business  Committee.  Her  offiee  provides  an 
annual  report  eard  on  the  status  of  the  federal  government  small  business  program.  In  the 
2003  edition,  twelve  of  the  twenty-one  federal  ageneies  seored  D  or  F,  ineluding  the 
Department  of  Defense,  whieh  improved  from  an  F  in  2002  to  a  D  in  2003  (Velazquez, 
2003:  12).  Velazquez  notes  that  “the  trend  is  apparent — while  the  federal  government’s 
buying  power  is  at  an  all-time  high,  its  small  business  goal  aehievement  is  at  an  all-time 
low”  (Velazquez,  2003:  15). 

In  a  separate  report.  Congresswoman  Velazquez  enumerates  a  top  ten  list  of  eontraets 
that  hurt  small  business.  Two  Air  Foree  eontracts  made  the  list.  In  this  report,  Velazquez 
concludes  that  the  Air  Force  plays  “a  shell  game  of  moving  multimillion-dollar  federal 
contracts  to  locations  where  another  federal  agency  may  agree  to  their  procurement 
strategy  of  choice”  and  by  doing  so  “the  Air  Force  is  excluding  small  businesses  from  the 
federal  contracting  arena”  (Velazquez,  2002:  5). 

In  addition,  acquisition  reform  initiatives  have  impacted  small  businesses.  A  report 
prepared  by  the  Office  of  Management  and  Budget  determined  that  since  1990,  the 
number  of  small  businesses  contracting  with  the  federal  government  had  been  reduced  by 
nearly  fifty  percent,  from  over  25,000  to  nearly  12,000  (Office  of  Management  and 
Budget,  2002:  5).  The  report  suggests  that  contract  bundling,  multiple  award  contracts 
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(MACs),  and  government-wide  acquisition  contracts  (GWACs),  each  a  product  of  reform 
efforts,  are  the  culprit  behind  the  reduced  small  business  base  for  the  federal  government 
as  a  whole  (Office  of  Management  and  Budget,  2002:  5). 

Efforts  to  reform  the  federal  procurement  process  span  over  forty  years  but 
accelerated  in  the  1990s.  Congress  began  a  major  effort  to  streamline  the  Department  of 
Defense’s  acquisition  process  with  the  Defense  Acquisition  Workforce  Improvement  Act 
of  1990.  As  budgetary  constraints  forced  the  Department  of  Defense  to  rethink  how  it 
contracts  for  supplies  and  services,  these  legislative  reforms  had  both  intended  and 
unintended  effects  on  small  businesses.  In  some  cases,  the  impact  of  the  reforms  on 
small  business  participation  is  unknown  and  immeasurable  due  to  the  lack  of  data 
collected  on  those  contracts.  New  reforms  are  continuously  proposed,  including  those 
that  are  a  product  of  the  current  effort  to  transform  the  military.  Each  of  these  reforms 
has  the  potential  to  either  positively  or  negatively  affect  the  success  of  the  small  business 
program. 

One  potential  reason  the  impact  of  acquisition  reform  efforts  on  small  business 
participation  is  unclear  is  that  the  success  of  the  small  business  program  is  measured  by  a 
single  metric  mandated  by  Congress.  This  single  metric  is  the  percentage  of  total  dollars 
awarded  to  small  businesses.  All  small  business  program  goals  are  based  on  this  single 
metric. 

Based  on  the  percentage  of  total  dollars  awarded  metric,  the  small  business  program 
of  the  Air  Eorce  has  had  declining  success  in  recent  years.  But  this  metric  is  greatly 
influenced  by  factors  such  as  the  overall  size  of  the  Air  Eorce  budget  and  may  not  be  the 
best  measure  of  small  business  program  success.  Although  this  metric  is  mandated  by 
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Congress,  additional  metrics  may  be  able  to  be  developed  that  could  present  a  clearer 
picture  of  the  state  of  small  business  programs. 

Small  businesses  are  a  vital  cog  in  the  American  economy.  They  are  also  an 
important  contributor  to  the  Air  Force  mission.  This  section  has  introduced  the 
importance  of  small  businesses,  discussed  the  legislative  background  of  the  small 
business  program,  explored  potential  problem  areas  in  current  small  business  contracting, 
and  examined  the  current  small  business  program  metric.  These  topics  lay  the 
groundwork  of  this  thesis  effort. 

Problem 

The  Air  Force  Small  Business  office  adopted  the  Air  Force  Small  Business  and 
HBCU/MI  (Historically  Black  Colleges  and  Universities/Minority  Institutions)  Program 
Plan  for  Fiscal  Years  2003-2007.  The  plan  “outlines  four  goals  that  are  designed  to 
create  a  solid  foundation  to  improve  the  efficiency  and  effectiveness  of  the  Air  Force  SB 
(small  business)  and  HBCU/MI  Program”  (Department  of  the  Air  Force,  2002).  The 
fourth  goal  of  the  Program  Plan  is  to  “enhance  the  effectiveness  of  the  SB  and  HBCU/MI 
Program  through  the  use  of  data  and  analysis”  (Department  of  the  Air  Force,  2002).  An 
integrated  product  team  consisting  of  both  small  business  personnel  and  other  military 
personnel  was  formed  to  determine  how  best  to  achieve  this  goal.  This  research  is  an 
outgrowth  of  that  team’s  efforts. 

The  purpose  of  this  research  is  to  assist  the  Office  of  Small  and  Disadvantaged 
Business  Utilization  in  determining  the  impact  of  acquisition  reform  on  small  business 
participation.  In  addition,  potential  metrics  will  be  identified  to  assist  that  Office  in 
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measuring  the  suceess  of  the  small  business  program.  Finally,  the  researeh  will  address 
the  potential  impaet  of  pending  transformation  reform  initiatives. 

Scope 

This  researeh  foeuses  speeifieally  on  the  small  business  program  of  the  United  States 
Air  Foree.  Only  those  aequisition  reform  initiatives  that  have  impaeted  Air  Foree 
eontraeting  proeedures  will  be  examined.  Speeifieally,  reforms  that  have  produeed 
measurable  results  will  be  the  foeus  of  the  study.  Contraet  data  for  the  time  period  of 
1990  to  the  present  will  be  examined  as  this  refleets  the  time  frame  of  the  acquisition 
reform  efforts  studied.  Because  of  the  limited  contract  data  available,  the  study  will  only 
focus  on  contracts  that  exceed  $25,000.  Above  this  dollar  amount,  contracting  officers 
must  provide  an  Individual  Contracting  Action  Report,  or  Department  of  Defense  Form 
350.  The  information  from  these  forms  for  the  Air  Force  are  stored  in  the  JOOl  database, 
which  is  a  subset  of  the  federal  government-wide  Federal  Procurement  Data  System 
(FPDS).  In  addition,  contracts  awarded  on  the  Federal  Supply  Schedule  (FSS)  or  General 
Services  Administration  (GSA)  schedules  are  not  included  because  contractor  size  has 
only  recently  begun  to  be  reported  for  these  contracts. 

Research  Questions 

1)  Have  acquisition  reform  initiatives  negatively  impacted  small  business 
participation  in  Air  Force  contracting? 

2)  Should  the  Air  Force  Small  Business  office  incorporate  the  measures  of  total 
small  business  contractors  and  total  new  small  business  contractors  into  their 
metrics? 
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Investigative  Questions 

What  benefits  are  derived  from  contracting  with  small  businesses? 

What  is  the  history  of  the  small  business  program  in  the  federal  government? 

What  metrics  measure  the  success  of  the  small  business  program? 

Are  those  effective  metrics  or  are  there  alternative  metrics  that  could  be  used? 

How  many  small  businesses  have  contracted  with  the  Air  Force  in  each  year  since 
the  Defense  Acquisition  Workforce  Improvement  Act  of  1990? 

How  many  new  small  businesses  have  contracted  with  the  Air  Force  in  each  year 
since  the  Defense  Acquisition  Workforce  Improvement  Act  of  1990? 

What  acquisition  reform  measures  have  been  enacted  and  what  are  their  perceived 
impact  on  small  business  contracting? 

Have  the  acquisition  reform  measures  impacted  the  number  of  small  businesses 
and  new  small  businesses  contracting  with  the  Air  Force? 

How  might  proposed  transformation  reforms  affect  future  small  business 
participation? 

Overview 

Chapter  I  provides  subject  matter  background  and  an  introduction  to  the  research  and 
investigative  questions.  Chapter  II  furnishes  a  literature  review  that  summarizes  the 
current  status  of  small  business  programs.  The  literature  review  focuses  on  the 
importance  of  small  business,  the  history  of  the  small  business  program,  acquisition 
reform  efforts  and  their  results,  with  a  specific  focus  on  contract  bundling  and 
performance-based  service  contracts,  and  an  overview  of  the  current  metric  and  potential 
metrics  to  measure  the  success  of  the  small  business  program.  Chapter  III  presents  the 
research  methodology  used  in  the  study  and  identifies  the  benefits  and  limitations  of  the 
methods  used.  Chapter  IV  provides  a  detailed  analysis  of  the  collected  data  and  the 


6 


resultant  findings.  Finally,  Chapter  V  provides  eonelusions  and  reeommendations  for 
future  researeh. 
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II.  Literature  Review 


The  purpose  of  this  chapter  is  to  provide  background  information  needed  to  answer 
the  research  questions  and  to  gain  an  understanding  of  the  research  objectives.  The  first 
section  provides  information  on  the  importance  of  small  businesses  to  the  national 
economy.  The  following  section  reviews  the  legislative  history  of  the  small  business 
program.  The  next  three  sections  examine  acquisition  reform  initiatives  affecting  Air 
Force  contracting  and  focus  specifically  on  two  products  of  that  reform,  contract 
bundling  and  performance-based  service  contracting.  A  look  at  current  reform  efforts, 
under  the  heading  transformation,  follows.  The  final  sections  provide  information 
needed  to  determine  if  two  metrics  proposed  by  this  research  are  appropriate  for  usage  by 
the  Air  Force  small  business  program. 

Contributions  of  Small  Business 

Small  businesses  contribute  to  the  national  economy  in  several  important  ways.  Acs 
and  Audretsch  (1998:  2)  point  out  that  small  businesses  and  their  owners  contribute  to  the 
national  economy  as  innovators,  entrepreneurs,  promoters  of  international 
competitiveness,  and  job  generators.  The  roles  of  small  businesses  as  innovators  and 
entrepreneurs,  in  particular,  led  Denes  (1997:  442)  to  declare,  “The  encouragement  of 
small  businesses  is  an  important  goal  of  many  modern  governments.”  This  section  will 
focus  on  small  business  as  a  source  of  innovation,  entrepreneurship,  and  job  creation.  It 
will  also  address  the  ways  that  small  businesses  contribute  to  the  economy  and  examine 
the  role  of  the  government  in  promoting  small  business  development. 
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Innovation. 


Small  firms  can  be  more  innovative  than  large  firms  beeause  “they  are  able  to  respond 
quickly  to  changing  market  demand,  are  organizationally  flexible,  and  have  efficient 
internal  eommunieations”  (Mogee,  2003:  3).  Small  firms  innovate  when  larger  firms  are 
more  eautious  beeause  “pioneering  new  markets  and  providing  information  in  small 
markets  are  often  inherently  risky  aetivities  and  may  yield  eeonomie  returns  that,  for 
larger  companies,  are  not  commensurate  with  risk”  (National  Aeademy  of  Engineering, 
1995:  37).  Intriguingly,  once  ereative  small  firms  assume  these  risks,  the  larger  firms  are 
foreed  to  be  more  inventive  as  well.  One  researcher  eoneludes  that  in  markets  dominated 
by  large  firms,  “the  small  firms  must  resort  to  a  strategy  of  innovation  in  order  to  remain 
viable”  (Acs  and  Audretsch,  1988:  687). 

Small  firms  show  an  additional  advantage  over  large  firms:  “small  firm  innovators  are 
extremely  effeetive  at  produeing  technieally  important  innovations. .  .small  firm 
innovations  are  more  than  twiee  as  likely  as  large  firm  innovations  to  be  extremely  high 
impaet”  (Chi  Researeh,  Ine.,  2003:  12). 

The  innovativeness  of  small  firms  ereates  jobs  in  addition  to  teehnologieal  advances. 
Entire  industries  may  be  ereated  through  the  suceessful  innovation  of  small  firms.  One 
researcher  eoneludes  that  most  growth  comes  from  “comparatively  younger  firms  in 
newer  industries”  and  that  most  “eeonomie  growth  eomes  from  new  small  businesses 
whieh  are  sueeessful  and  grow  eventually  to  be  large”  (Terleekyj,  1999:  3). 

Small  business  plays  a  vital  role  in  the  development  of  new  teehnology.  As  the  Air 
Eorce  has  beeome  more  technologieally  advanced  and  focused,  the  need  to  eneourage 
small  business  development  and  the  resultant  innovativeness  has  inereased  as  well. 
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Entrepreneurship, 

Creating  an  economic  climate  that  promotes  entrepreneurship  is  a  key  element  in 
promoting  both  innovation  and  small  business.  Creating  such  an  environment  is  vital  to 
the  federal  government’s  small  business  programs.  “A  cooperative  project  between  an 
innovative  small  business  and  a  bureaucratically  attuned  partner  can  lead  to  positive 
outcomes  for  the  small  business  project  team  and  many  publics”  (Brannen  and  Card, 
1985;  49). 

The  benefits  of  entrepreneurship  are  numerous.  Entrepreneurship  allocates  resources 
to  new  competitive  uses,  it  “introduces  new  knowledge  or  combines  old  knowledge  in 
radically  novel  ways  to  improve  economic  performance,”  as  well  as  creating  human 
capital,  “the  basic  source  of  innovative  activity”  (Karlsson  and  Karlsson,  2002:  179). 
Promoting  entrepreneurship  encourages  innovative  behavior  and  economic  growth. 
“Entrepreneurship  is  the  best  instrument  to  secure  that  resources  that  have  become  idle 
are  put  into  productive  work  as  soon  as  possible”  (Karlsson  and  Karlsson,  2002). 

By  facilitating  an  environment  that  promotes  the  entrepreneurial  spirit,  the  Air  Eorce 
serves  as  agent  for  achieving  an  economic  equilibrium  in  addition  to  promoting 
innovation.  Promoting  small  business  facilitates  such  an  environment. 

Job  Creation. 

Small  business  plays  an  important  function  as  a  creator  of  jobs  and  also  serves  as  an 
entrance  to  the  workforce  for  many  workers.  Studies  reveal,  “Over  a  4-5  year  period, 
about  three-fourths  (75-80%)  of  new  jobs  have  been  shown  to  come  from  small  firms. 
About  30-40%  of  these  new  jobs  have  come  from  the  births  of  new  firms  and  60-70% 
from  rapidly  expanding  ‘gazelle’  firms”  (Office  of  Advocacy,  2000;  13).  “Gazelles”  is  a 
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term  coined  by  David  Birch,  which  he  defined  as  a  firm  that  grows  “at  least  20%  a  year 
for  four  years  from  a  base  of  at  least  $100,000  in  revenues”  (“  The  Gazelle  Theory”, 
2001:  29).  An  example  of  small  business’  impact  on  the  labor  market  is  that  in  2000, 
small  businesses  added  “a  net  total  of  2,505,712  employees;  they  represented  75  percent 
of  net  non-farm  employment  change  in  the  United  States”  (Office  of  Advocacy,  2003:  1). 
(The  Department  of  Labor  measures  farm  and  non- farm  employment  separately.  The 
non-farm  labor  totals  are  what  are  typically  reported  in  the  news  media.)  Clearly,  small 
businesses  are  a  major  force  in  the  labor  market. 

Small  firms  also  fill  a  critical  niche  in  the  job  market  as  “the  essential  mechanism  by 
which  millions  enter  the  economic  and  social  mainstream  of  American  society”  (Office 
of  Advocacy,  1998:  4).  Small  business  is  “especially  crucial  in  determining  the  job 
opportunities  for  low-skill  workers”  (Fitzgerald  and  Ribar,  2001:  1).  What  distinguishes 
small  businesses  in  the  labor  market  is  their  ability  “to  utilize  secondary,  or  less 
attractive,  resources  in  the  marketplace”  (Robbins  and  others,  2000:  295).  These 
secondary  resources  are  defined  as  “first  time  entrants  into  the  job  market,  the  long-term 
unemployed,  individuals  at  low  educational  levels,  part-time  employees,  women,  certain 
minorities,  immigrants,  short-term  workers,  the  previously  self-employed  and  workers 
under  the  age  of  twenty”  (Robbins  and  others,  2000:  295). 

Contributions  to  the  Economy, 

Small  business  promotes  innovation  and  entrepreneurship  and  creates  jobs.  Each  of 
these  functions  is  an  important  contribution  to  the  economy.  However,  small  business 
contributes  to  the  economy  in  other  ways  as  well.  Small  business  contributions  to  the 
economy  can  be  viewed  in  two  ways:  quantitatively  and  qualitatively.  From  a 
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quantitative  perspective,  statistical  analysis  can  be  used  to  demonstrate  the  contributions 
of  small  businesses.  The  qualitative  view  identifies  attributes  that  enable  small 
businesses  to  succeed. 

The  Small  Business  Administration  defines  small  businesses  as  those  that  employ 
fewer  than  500  people.  Based  on  that  definition,  99.9%  of  all  firms  in  the  United  States 
are  small  businesses,  and  these  businesses  “accounted  for  56.5%  of  the  nation’s  private 
sector  workforce”  (Office  of  Advocacy,  2000;  7).  In  addition,  47%  of  all  sales  revenue 
comes  from  small  firms  and  these  firms  contributed  over  52%  of  non-farm  Gross 
Domestic  Product  in  1992  (Office  of  Advocacy,  2000:  7). 

Small  firms  are  more  flexible  than  large  firms  and  thus  can  respond  faster  to  changes 
in  the  environment.  This  flexibility  ensures  that  “the  arrival  of  any  major  new  technology 
should  favor  young,  small  firms — firms  that  tend  to  have  a  nonhierarchical  structure, 
fewer  unionized  workers,  and  fewer  outdated  management  practices”  (Jovanovic,  2001; 
54). 

Joel  Popkin  addresses  two  factors  that  affect  small  business  contribution  in  the 
economy.  According  to  Popkin,  “the  change  within  each  industry  sector  of  the 
importance  of  small  business  .  .  .  and  the  shift  in  the  economy  toward  the  industries  in 
which  small  businesses  are  providing  the  highest  share  of  outpuf  ’  both  help  determine  the 
long-term  role  of  small  business  in  the  economy  (Popkin,  2001;  2). 

Government  Role. 

Small  business  contributes  to  innovation,  entrepreneurship,  job  creation,  and 
economic  growth,  but  the  involvement  of  the  government  in  protecting  and  promoting 
small  business  is  a  topic  of  substantial  debate.  Some  researchers  have  concluded  that 
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government  involvement  is  essential  to  the  well  being  of  small  business.  Lemer 
questions  government  promotion  of  small,  high  technology  firms,  but  also  acknowledges 
two  rationales  for  government  involvement.  First,  small  firms  lack  the  resources  needed 
for  research  and  development  (R  &  D),  and  government  investment  can  produce 
substantial  returns.  Second,  the  government  can  serve  as  a  certifier  of  a  small  firm’s 
potential  success  and  increase  the  firm’s  ability  to  attract  venture  capital  by 
demonstrating  to  other  potential  investors  the  legitimacy  of  the  small  business  (Lerner, 
1999:  286).  An  unintended  benefit  the  government  receives  from  this  investment  is  R  & 
D  “spillover,”  in  which  the  increased  innovativeness  of  one  firm  in  a  market  leads  other 
firms  (both  large  and  small)  to  increase  their  investment  in  R  &  D  (Lemer,  1999:  290). 

A  recurring  concern  over  small  business  set-asides  by  the  federal  government  is  that 
they  reduce  competition  and  increase  costs  for  the  government.  But  Denes’  (1997:  444) 
research  demonstrates  that  the  number  of  bidders  is  not  reduced  if  the  procurement  is  set- 
aside  for  small  business  and  that  “small  business  set-asides  do  not  lead  to  higher  cost  of 
contracted  services.” 

Summary  of  Small  Business  Contributions. 

Small  businesses  fill  an  important  role  in  the  economy.  They  are  a  source  of 
innovation  and  promote  the  entrepreneurial  spirit.  They  provide  over  half  of  the  jobs  in 
the  nation  and  serve  an  important  function  as  an  entry  point  into  the  labor  market  for 
many  workers.  Small  businesses  are  flexible  and  provide  an  essential  boost  to  economic 
growth.  The  federal  government  encourages  and  protects  small  businesses  in  order  to 
receive  these  benefits,  and  does  so  at  a  limited  cost.  The  evidence  of  government 
encouragement  and  protection  is  a  long  history  of  legislation  and  regulation. 
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Small  Business  Statutes  and  Regulations 

The  small  business  program  has  been  developed  and  refined  over  the  past  fifty  years 
by  Congress  and  the  President  through  legislation.  The  eurrent  status  of  the  program  ean 
be  eaptured  in  a  review  of  the  appropriate  regulations  that  apply  to  the  federal 
government  in  general  and  speeifieally  to  the  Department  of  Defense  (DoD)  and  the  Air 
Foree. 

Statutes. 

For  over  fifty  years,  laws  passed  by  Congress  have  emphasized  the  importanee  of 
small  businesses.  The  following  seetion  highlights  laws  relating  to  Federal  procurement 
and  small  businesses,  presented  in  chronological  order: 

*  The  Small  Business  Act  of  1953  (Public  Law  85-536)— Established  the  Small 
Business  Administration  (SBA)  as  an  independent  agency  within  the  Federal 
government.  In  part  the  Act  reads: 

It  is  the  declared  policy  of  the  Congress  that  the  Government 
should  aid,  counsel,  assist,  and  protect,  insofar  as  it  is  possible,  the 
interests  of  small-business  concerns  in  order  to  preserve  free  competitive 
enterprise,  to  ensure  that  a  fair  proportion  of  the  total  purchases  and 
contracts  ...  for  the  Government  be  placed  with  small  business  enterprises 
.  .  .  and  to  maintain  and  strengthen  the  overall  economy  of  the  Nation. 

(United  States  Congress,  1953:  Sec  2  (a)) 

*  Amendment  to  the  Small  Business  Act  (Public  Law  94-305)— Created  an  Office 
of  Advocacy  within  the  SBA.  The  law  requires  the  Office  of  Advocacy  to  “evaluate 
Federal  and  business  efforts  to  assist  minority  enterprises”  and  to  “propose  changes  in 
practices  of  other  agencies  to  assist  small  business”  (United  States  Congress,  1976:  Title 
II).  The  Office  of  Advocacy  serves  as  “an  independent  voice  for  small  business  in  the 
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formulation  of  public  policy  across  the  entire  federal  government”  (Office  of  Advoeaey, 
2000:  4). 

*  Amendment  to  the  Small  Business  Act  (Public  Law  95-507)— Changed  the  Small 
Business  Aet  to  provide  “maximum  praetieable  opportunity”  for  “small  business 
eoneerns  owned  and  eontrolled  by  soeially  and  economieally  disadvantaged  individuals” 
(United  States  Congress,  1978:  Title  II).  The  law  requires  eontraetors  who  submit  bids 
on  eontracts  exeeeding  $10,000  to  submit  a  subeontracting  plan  and  requires  federal 
agency  heads  “to  establish  goals  for  small  business  participation  in  specified  federal 
proeurement  eontracts”  (United  States  Congress,  1978:  Title  II).  A  final  provision  of  the 
law  ereated  in  eaeh  federal  ageney  an  Offiee  of  Small  and  Disadvantaged  Business 
Utilization. 

*  Small  Business  Innovation  Development  Aet  of  1982  (Publie  Law  97-219)— 
Created  the  Small  Business  Innovative  Research  (SBIR)  program  requiring  federal 
ageneies  to  set  goals  for  reaehing  research  and  development  agreements  with  small 
businesses.  By  requiring  set-asides,  the  law  intended  to  “strengthen  the  role  of  the  small, 
innovative  firm  in  federally  funded  research  and  development”  (United  States  Congress, 
1982:  Title  I). 

*  National  Defense  Authorization  Aet  for  Fiscal  Year  1987  (Publie  Law  99-661)— 
Established  speeifie  goals  for  the  DoD  in  three  areas:  (1)  small  business  eoneerns;  (2) 
Historieally  Blaek  Colleges  and  Universities  (HBCUs);  and  (3)  minority  institutions 
(United  States  Congress,  1986:  Title  IX,  Part  B). 

*  Small  Business  Competitiveness  Demonstration  Program  Aet  of  1988  (Publie 
Law  100-656)— Required  quarterly  reporting  of  federal  agency  small  business 
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participation  goal  achievement  and  directed  federal  agency  heads  to  “implement  a 
program  to  inerease  small  business  partieipation  in  ageney  aequisition  of  seleeted 
produets  and  serviees  in  industry  categories  historieally  low  in  partieipation”  (United 
States  Congress,  1988;  Title  VII,  Part  B). 

*  National  Defense  Authorization  Act  for  Fiscal  Year  1991  (Public  Law  I0I-5I0)- 
-Introduced  two  relevant  topies.  First,  it  required  the  Secretary  of  Defense  to  establish 
speeific  goals  for  contracting  to  HBCUs  and  minority  institutions  “in  order  to  inerease 
participation  of  such  colleges  and  universities  in  defense  research  programs”  (United 
States  Congress,  1990:  Title  VIII,  Part  D).  Seeond,  the  law  created  the  Mentor-Protege 
Program  “to  provide  incentives  to  major  DoD  contractors  (mentors)  to  help 
disadvantaged  small  businesses  (proteges)  perform  as  subcontraetors  and  suppliers  under 
DoD  and  other  Government  contracts”  (United  States  Congress,  1990:  Title  VIII,  Part  D). 

*  Small  Business  Teehnology  Transfer  Act  of  1992  (Public  Law  102-564)— 
Created  the  Small  Business  Technology  Transfer  (STTR)  program  that  reserves  a  set 
pereentage  of  each  federal  ageney’s  research  and  development  effort  for  small  businesses 
in  a  format  similar  to  the  SBIR  program  (United  States  Congress,  1992;  Title  II). 

*  Federal  Aequisition  Streamlining  Aet  of  1994  (Public  Law  103-355)— Reserved 
all  federal  procurements  between  $2,500  and  $100,000  for  small  businesses.  It 
established  goals  for  awards  to  small  businesses  owned  and  eontrolled  by  women.  It  also 
authorized  ageney  heads  to  restrict  competition  on  some  contracts  to  small  businesses 
owned  by  soeially  and  economically  disadvantaged  individuals  (United  States  Congress, 
1994:  Title  IV,  Section  4004,  Title  VII,  and  Title  VII,  Seetion  7102). 
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*  Small  Business  Reauthorization  Act  of  1997  (Public  Law  105-135)— Established 

two  new  programs:  the  Historically  Underutilized  Business  Zone  (HUBZone)  program 

and  the  Service  Disabled  Veteran’s  program.  The  two  programs  shared  the  same  goal:  to 

increase  federal  agency  contracting  with  each  set  of  small  businesses.  This  act  also 

amended  the  Small  Business  Act  by: 

Requiring  each  federal  agency  to  (1)  foster  the  participation  of  small 
businesses  as  prime  contractors,  subcontractors,  and  suppliers;  (2) 
structure  its  contracting  requirements  to  facilitate  competition  by  and 
among  small  businesses;  and  (3)  avoid  the  unnecessary  and  unjustified 
bundling  of  contracts  that  precludes  small  business  participation  as  prime 
contractors.  (United  States  Congress,  1997:  Title  IV,  Subtitle  B) 

The  preceding  legislation  is  only  some  of  the  legislation  guiding  federal  agency 
contracting  in  relation  to  small  businesses.  In  the  last  fifty  years.  Congress  has  modified 
almost  annually  the  rules  of  the  federal  small  business  program.  Specific  Acts  have 
changed  the  focus  of  the  socio-economic  programs  established  by  Congress,  but  the  basic 
intent  of  protecting  small  businesses  and  encouraging  their  growth  has  remained  constant. 
Congressional  actions  to  protect  and  promote  small  business  are  the  foundation  of  each 
federal  agency’s  small  business  program. 

Regulations. 

The  Federal  Acquisition  Regulation  (FAR)  provides  the  guidelines  for  the 
procurement  of  supplies  and  services  for  the  federal  government.  Each  agency  within  the 
government  provides  a  supplement,  including  the  DoD  Federal  Acquisition  Regulations 
(DFARs).  Within  the  DoD,  each  service  maintains  its  own  supplement,  such  as  the  Air 
Force  Federal  Acquisition  Regulations  (AFFARs).  Within  the  Air  Force,  separate 
commands  also  provide  their  own  supplements,  such  as  the  Pacific  Air  Forces  Federal 
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Acquisition  Regulations  (PAFFARs).  These  supplements  do  not  ehange  or  eontradiet  the 
FAR  laws  but  instead  they  provide  further  insight  and  explanation  for  eireumstanees 
partieular  to  eaeh  organization.  The  supplements  “shall  not,  except  as  required  by  law  or 
as  provided  in  Subpart  1 .4  (Deviations  from  the  FAR),  eonfliet  or  be  ineonsistent  with 
FAR  eontent”  (Federal  Aequisition  Regulation,  2003:  FAR  1.304  (b)(2)). 

In  the  FAR,  Part  19  is  titled  Small  Business  Programs.  Part  19  provides  federal 
employees  speeifie  guidance  for  implementing  the  laws  passed  by  Congress.  The  poliey 
seetion,  subpart  19.2,  opens  by  deelaring,  “It  is  the  poliey  of  the  government  to  provide 
maximum  practicable  opportunities  (emphasis  added)  in  its  aequisitions  to  small 
business”  (Federal  Aequisition  Regulation,  2003:  FAR  19.201).  Part  19  outlines  the 
policies  of  the  federal  government  in  regards  to  small  business.  From  speeifying  the 
requirement  for  eaeh  ageney  to  have  an  Offiee  of  Small  and  Disadvantaged  Business 
Utilization  to  providing  direetion  on  how  to  loeate  small  business  sourees  and 
determining  fair  market  priees,  FAR  Part  19  spells  out  the  requirements  established  in  the 
Small  Business  Aet  and  other  small  business  legislation  that  impaet  government 
eontracting  personnel. 

In  the  DFARs,  the  relevant  seetion  is  Part  219.  It  provides  further  guidanee  on 
performing  the  speeifie  actions  associated  with  the  FAR.  It  is  within  the  DFARs,  for 
example,  that  the  speeifie  responsibilities  of  the  small  business  speeialist  are  defined. 

One  of  these  responsibilities  is  “reviewing  and  making  recommendations  for  all 
aequisitions  over  $10,000”  (Department  of  Defense  Federal  Aequisition  Regulations, 
2003:  DFARS  219.201  (d)(10)(A)).  The  DFARs  serves  to  define  items  that  apply 
speeifieally  to  the  military  in  regards  to  the  small  business  program. 
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Summary  of  Small  Business  Statutes  and  Regulations, 

Congress  and  the  president  determine  what  aetions  to  take  regarding  small  business 
legislation.  Historically,  many  of  these  actions  affect  the  way  federal  agencies  perform 
their  contracting  function.  When  laws  pass  that  change  the  regulations,  then  the 
responsible  agencies  (such  as  the  Department  of  Defense,  etc.)  must  review  and  amend 
their  regulations  to  implement  the  required  changes.  As  in  other  areas  of  acquisition,  the 
FAR  and  its  supplements  serve  as  a  living  document  to  capture  the  most  current  status  of 
small  business  legislation  in  their  regulations. 

Acquisition  Reform 

While  the  Congressional  efforts  to  support  small  business  have  been  numerous,  the  list 
of  attempts  to  reform  the  DoD  procurement  system  is  just  as  lengthy.  Such  efforts  can  be 
traced  back  to  the  Armed  Services  Procurement  Act  of  1947,  which  was  an  attempt  to 
standardize  purchasing  methods  between  the  various  military  services  (Battershell,  1999; 
24).  This  section  will  focus  on  reforms  made  in  the  1990s  because  “reforming 
government  to  reduce  cost  and  improve  operations  has  been  a  major  theme  throughout 
the  1990s”  (Chinworth,  2000:  165).  The  purpose  of  this  section  is  to  briefly  review  three 
legislative  actions  and  a  Supreme  Court  ruling  that  directly  affect  small  business 
participation  in  Air  Force  contracting,  followed  by  a  review  of  how  these  events  are 
perceived  to  have  impacted  small  business  participation. 

Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  of  1990. 

DAWIA  is  not  an  acquisition  reform  initiative  per  se,  because  it  does  not  change  any 
of  the  acquisition  processes.  DAWIA  created  an  acquisition  corps  for  each  component  of 
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DoD  and  established  eligibility  and  seleetion  eriteria  for  membership  in  that  eorps.  It 
also  established  a  training  program  that  emphasized  the  importanee  of  an  edueated 
aequisition  workforee  (United  States  Congress,  1990;  Title  XII).  DAWIA  ereated  “one 
of  the  most  eomprehensive  eareer  development  programs  in  the  federal  government, 
designed  to  ensure  that  aequisition  personnel  have  the  knowledge  and  skills  neeessary  to 
perform  effeetively”  (“Under  Seeretary  of  Defense  (Aequisition  and  Teehnology)  Sets 
New  Standards  for  the  Defense  Aequisition  Workforee”,  1999:  10). 

Continual  reform  of  the  training  and  edueation  requirements  of  personnel  is  neeessary 
beeause  the  “aequisition  workforee  of  the  future  must  understand  and  internet  effeetively 
with  the  eommereial  seetor,  leverage  best  business  praetiees  and  teehnologieal  advanees 
for  eontinuing  proeess  improvement,  and  possess  strong  management  and  leadership 
eompeteneies”  (Cohen,  1998:  22).  A  more  edueated  workforee  is  also  “the  key  to 
suoeessful  aequisition  reform  implementation”  (Cooper,  2002:  37). 

Federal  Acquisition  Streamlining  Act  (FAS A)  of  1994, 

FASA  acted  upon  recommendations  from  the  National  Performance  Review  and  the 
Advisory  Panel  on  Streamlining  and  Codifying  Acquisition  Laws  and  “generated  the 
broadest  and  most  far-ranging  changes  to  the  government  procurement  system  in  almost  a 
decade”  (Tolan  Jr.,  1998,  89).  Among  its  numerous  initiatives,  FASA  contained  four 
major  provisions  that  directly  impact  small  business. 

First,  FASA  allowed  for  the  creation  of  multiple-award  contracts  (MACs),  which  are 
used  to  allow  for  the  acquisition  of  the  same  item  from  multiple  firms  (United  States 
Congress,  1994:  Title  V,  Subtitle  D).  MACs  are  “designed  to  provide  agencies  with  a 
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great  deal  of  flexibility  in  buying  goods  or  services  while  minimizing  the  burden  on 
government  contracting  personnel  to  administer  contracts”  (Hecker,  2001). 

Second,  FASA  created  the  label  of  micropurchase  for  those  acquisitions  of  $2,500  or 
less  (United  States  Congress,  1994;  Title  IV,  Subtitle  D).  For  these  purchases, 
“government  buyers  do  not  have  to  obtain  competition  and  are  not  required  to  purchase 
goods  or  services  from  small  businesses”  (Hecker,  2001).  In  addition,  FASA  raised  the 
Simplified  Acquisition  Threshold,  “the  ceiling  value  at  which  individual  procurement 
officers  can  make  discretionary  purchases  without  resorting  to  public  procurement 
competitions”  (Chinworth,  2000:  166)  to  $100,000,  and  reserved  these  contracts 
exclusively  for  small  business. 

FASA  also  signaled  a  major  shift  in  federal  procurement  in  moving  from  items 
purchased  using  military  standards  or  specifications  to  commercial  items.  FASA  created 
“a  preference  under  current  law  for  procurement  of  commercial  items  (or 
nondevelopmental  items  other  than  commercial  items  if  commercial  items  suitable  to 
meet  the  agency’s  needs  are  not  available)”  (United  States  Congress,  1994;  Title  VIII). 

Finally,  FASA  introduced  performance-based  service  contracting  (PBSC)  “as  a 
method  of  reducing  acquisition  costs  and  improving  contractor  performance  by 
encouraging  innovative  approaches  to  conducting  the  work  within  desired  outcomes” 
(Cooper,  2002;  20).  PBSC,  like  the  move  to  commercial  items,  reflects  the  reformers 
efforts  to  be  less  prescriptive  in  the  acquisition  process.  It  provides  both  contracting 
personnel  and  prospective  contractors  greater  flexibility  in  meeting  agency  needs. 
Performance-based  service  contracting  will  be  discussed  in  greater  detail  later  in  this 
chapter. 
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Clinger-Cohen  Act  of  1996  (Clinger-Cohen), 

The  Clinger-Cohen  Act  refers  to  two  acts  passed  under  the  DoD  Authorization  Act  of 
1996,  the  Federal  Acquisition  Reform  Act  (FARA)  of  1996  and  the  Information 
Technology  Management  Reform  Act  (ITMRA)  of  1996  (United  States  Congress,  1996: 
Division  D). 

FARA  furthered  the  reforms  made  in  FASA.  It  established  commercial  off-the-shelf 
(COTS)  items  as  a  subset  of  commercial  items  and  “is  even  more  generous  in  exempting 
COTS  items  from  federal  procurement  laws  than  FASA  was  in  exempting  ‘commercial’ 
acquisitions  from  the  ordinary  bureaucracy  of  the  government  purchasing  system”  (Tolan 
Jr.,  1998:  77).  FARA  also  provided  “greater  flexibility  to  agencies  in  determining  who 
may  make  purchases  of  $2,500  or  less  without  competition”  (Hecker,  2001). 

ITMRA  required  “all  federal  agencies  to  link  their  technology  plans  and  information 
technology  use  to  the  agency  missions  and  goals”  (Chinworth,  2000:  166).  ITMRA 
removed  the  central  authority  of  the  General  Services  Administration  (GSA)  to  make 
information  technology  purchases  and  “provided  for  the  use  of  multiagency  contracts  and 
what  have  become  known  as  government  wide  agency  contracts  (GWACs)  for  federal 
agencies  to  access  each  other’s  information  technology  contracts”  (Hecker,  2001). 

Adarand  Constructors,  Inc.  v.  Pena  (Adarand). 

The  Adarand  case  is  significant  because  of  its  impact  on  one  part  of  the  Air  Force 
small  business  program,  that  part  dealing  with  small,  disadvantaged  businesses  (SDBs). 
Adarand  is  a  small  business  that  bid  as  a  subcontractor  to  provide  guardrails  for  the 
federal  highway  system  in  Colorado,  but  lost  the  contract  because  of  additional 
compensation  provided  to  the  prime  contractor  to  subcontract  with  SDBs  (Wallace  and 
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Schooner,  1997;  3).  The  Supreme  Court  concluded,  “Government  contracting  initiatives 
to  favor  small  and  disadvantaged  businesses  would  have  to  overcome  strict  scrutiny” 
(Tolan  Jr.,  1998:  75).  The  striet  scrutiny  test  requires  the  government  to  determine 
“whether  the  interest  cited  by  the  government  as  its  reason  for  injeeting  the  consideration 
of  race  is  sufficiently  eompelling  to  overcome  the  suspicion  that  racial  characteristics 
ought  to  be  irrelevant”  (Wallaee  and  Sehooner,  1997:  6)  . 

In  response  to  Adarand  changes  were  made  to  the  FAR.  According  to  Welch,  “the 
government’s  implementation  of  the  Supreme  Court’s  ruling  took  years  and  generated 
more  than  150  pages  of  policy  and  regulation”  (Welch,  2000:  107).  The  changes  were 
made  with  the  intent  of  “trying  to  tailor  the  remedies  more  narrowly  so  that  the  SDB 
program  will  survive  striet  scrutiny”  (Tolan  Jr.,  1998:  97). 

Perceived  Effects  on  Small  Business. 

There  are  several  reasons  why  aequisition  reform  took  place.  Chinworth  outlines 
several  of  these: 

Acquisition  reform  is  intended  to  reduce  proeurement  costs  and 
complexity  through  streamlined  procurement  proeesses,  greater  autonomy 
in  deeision-making  at  lower  levels  of  government,  redueed  reliance  on 
military  standards  and  specifieations  and  greater  utilization  of  commercial 
standards  and  business  practiees.  Another  goal  of  aequisition  reform  has 
been  to  bring  about  more  commonality  in  production  programs  by 
eliminating  DoD-specific  requirements  in  the  management  of  those 
programs.  (Chinworth,  2000:  167) 

The  reforms  made  have  produced  an  aequisition  culture  that  is  “more  team-based,  value- 
based,  results-driven,  project-oriented,  and  relationship-foeused”  (Burman,  2000;  69). 

But  some  question  the  impaet  of  these  reforms  on  small  business.  One  deelares  the 
reforms  a  Trojan  horse  to  small  business  and  states  that  they  work  “against  the 


23 


entrepreneurial  instinets  and  innovative  drives  of  many  small-  to  medium-sized  firms;  a 
set  of  reforms  that  arbitrarily  exeludes  many  of  these  firms  from  the  opportunity  to 
eompete  for  federal  dollars”  (Campbell,  1997:  40).  Another  feels  that  “regulations 
intended  to  streamline,  simplify,  and  speed  up  government  proeurement  are  well  and 
good,  but  small  businesses  worry  that  these  reforms  may  be  shutting  them  out  of  the 
federal  marketplaee”  (Burman,  2001:  78).  Senator  John  Kerry,  D-Mass,  deelared  in 
August  of  2002  that,  “Proeurement  reform  of  the  early  1990s  has  not  proteeted  small 
business.  We  need  a  major  ehange  in  how  we  are  thinking  about  this”  (Burman,  2002: 
61).  The  bigger  issue  may  be  that  “many  of  these  wonderfully  effeetive  and  effieient 
aequisition  reforms  are  seldom  applied  with  the  intent  of  benefiting  small  businesses” 
(Weleh,  2000:  107). 

For  example,  sinee  DAWIA,  the  DoD  “has  redueed  the  aequisition  workforee  by  42 
pereent”  (Cohen,  1998:  21).  Burman  feels  this  exaeerbates  the  negative  impaet  of 
reforms  on  small  business  beeause  “these  offieials  are  hard-pressed  to  keep  up  with  the 
rules  and  various  protoeols  assoeiated  with  eaeh  legislative  requirement  and  reporting 
tools  that  are  frequently  weak  or  nonexistent”  (Burman,  2001:  78). 

The  inereased  use  of  MACs  and  GWACs  has  raised  eoneems  that  large  eontraets 
squeeze  out  small  business.  Some  believe  that  “these  types  of  eontraet  vehieles  ean 
diminish  the  ability  of  small  businesses  to  eompete  for  federal  eontraets  sinee  they 
potentially  ean  eonsolidate  requirements,  whieh  small  businesses  may  not  have  the 
eapaeity  to  meet”  (Heeker,  2001). 

The  ehange  to  ereate  a  mieropurehase  level  of  $2,500  and  not  require  those  purehases 
to  go  to  small  business  also  may  negatively  affeet  small  businesses.  It  is  possible  that 
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“buyers  making  micropurchases  may  be  less  likely  to  seek  small  businesses  for  these 
purchases”  (Hecker,  2001).  An  opposing  view  is  that  because  micropurchases  are 
“typically  easy  to  satisfy  by  award  to  local  small  or  small  disadvantaged  businesses  .  .  . 
there  need  be  no  adverse  impact  on  small  business  due  to  this  change”  (Tolan  Jr.,  1998: 
90).  However  determining  which  viewpoint  is  accurate  is  difficult  because  most  of  these 
purchases  are  made  with  the  government  purchase  card,  and  “ultimately,  there  is  no  good 
way  to  determine  how  much  of  the  credit  card  business  is  going  to  small  companies” 
(Burman,  2001:  78)  because  part  of  the  reason  the  government  purchase  card  is  used  is  to 
reduce  paperwork.  It  is  this  very  paperwork  that  has  been  eliminated  which  is  needed  to 
effectively  track  small  business  dollars  in  micropurchases. 

Finally,  the  Adarand  case  created  a  great  concern  among  SDBs  that  their  work  with 
the  government  would  be  reduced.  The  reforms  made  assuaged  most  of  those  fears,  but 
“SDBs  remain  concerned  that  the  affirmative  action  measures  can  be  curtailed  or 
eliminated  based  upon  the  success  of  SDBs  in  obtaining  government  work  within  certain 
industries”  (Wallace  and  Schooner,  1997:  10). 

Summary  of  Acquisition  Reform. 

Several  major  acquisition  reforms  in  the  1990s  completely  reshaped  the  government 
acquisition  process  and  attempted  to  streamline  the  acquisition  process.  Many  of  the 
reforms  have  created  concern  among  small  businesses  that  they  could  be  negatively 
affected  or  even  prevented  from  participating  in  the  federal  contracting  process.  The 
following  sections  will  look  more  closely  at  two  byproducts  of  the  reforms  that  may  have 
impacted  small  business  participation:  bundled  contracts  and  performance-based  service 
contracts. 
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Contract  Bundling 


Contract  bundling  is  not  specifically  a  product  of  the  aequisition  reform  legislation  of 
the  1990s,  but  it  is  “an  aequisition  practiee  that  received  a  lot  of  attention  in  the  1990s 
and  is  often  assoeiated  with”  the  reform  legislation  (Heeker,  2001).  Contracts  are 
bundled  beeause  it  “makes  eontraet  oversight  easier  for  ageneies,  but  it  frequently  makes 
new  awards  so  big  that  small  businesses  can’t  compete  for  the  work”  (Burman,  2002:  61). 
The  purpose  of  this  seetion  is  to  review  the  history  of  eontraet  bundling  and  recap  what 
research  has  been  done  on  its  impaet  on  small  business. 

Definition. 

The  Small  Business  Reauthorization  Aet  of  1997  ineluded  a  definition  for  a  bundled 
eontraet.  That  definition  is  included  in  Chapter  2  of  the  FAR.  It  reads: 

Bundling  means- 

(1)  Consolidating  two  or  more  requirements  for  supplies  or  serviees, 
previously  performed  under  separate  smaller  eontraets,  into  a 
solicitation  for  a  single  contract  that  is  likely  to  be  unsuitable  for 
award  to  a  small  business  eoncern  due  to- 

(i)  The  diversity,  size,  or  speeialized  nature  of  the  elements  of 
the  performance  specified; 

(ii)  The  aggregate  dollar  value  of  the  antieipated  award; 

(iii)  The  geographieal  dispersion  of  the  eontraet  performance 
sites;  or 

(iv)  Any  eombination  of  the  faetors  described  in  paragraphs 
(l)(i),  (ii),  and  (iii)  of  this  definition.  (Federal  Aequisition 
Regulation,  2003:  2.101(b)) 

Past  Research, 

Eagle  Eye  Publishers  produeed  the  first  major  study  to  attempt  to  measure  the  impaet 
of  eontraet  bundling  on  small  business  in  June  of  1997.  This  study  examined  federal 
eontraet  data  for  the  fiseal  years  1991  to  1995.  The  report  dealt  with  the  lack  of  a 
definition  of  a  bundled  contract  by  identifying  Candidate  Bundled  Contracts  (CBCs), 
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which  were  defined  as  “any  eontract  showing  multiple  SIC  (Standard  Industrial 
Classification)  codes,  multiple  types  of  contraet  codes  or  multiple  plaees  of  performanee” 
(Eagle  Eye  Publishers,  1997:  9).  Based  on  this  definition,  the  study  found  that  bundling 
was  oceurring  at  an  increasing  rate.  In  addition,  the  number  of  small  businesses  and  new 
small  businesses  receiving  contract  awards  declined  as  well.  The  study  concludes,  “The 
data  show  indisputably  that  fewer  and  larger  contraets  are  being  won  by  fewer  and  larger 
companies.  Eurthermore,  it  appears  small  businesses  are  being  squeezed  out  of  the 
federal  marketplaee  as  these  trends  occur”  (Eagle  Eye  Publishers,  1997:  17). 

The  United  States  General  Aecounting  Office  (GAO)  published  a  report  in  Mareh 
2000  on  the  extent  and  effect  of  contract  bundling.  Using  the  definition  provided  by  the 
Small  Business  Reauthorization  Act  of  1997,  the  GAO  concluded  that  many  requirements 
had  been  consolidated  but  did  not  meet  the  legal  definition  of  bundling.  In  fact,  in 
reviewing  three  procurement  centers,  the  report  “identified  one  eontract  bundling  case  of 
all  the  contract  consolidations  we  reviewed”  (General  Accounting  Office,  2000:  15).  The 
GAO  study  was  limited  as  “there  is  very  little  data  on  the  extent  of  contraet  bundling 
government  wide  and  its  effect  on  small  business”  (General  Aceounting  Office,  2000:  3). 

In  September  2000,  Eagle  Eye  published  an  updated  report,  this  one  spanning  from 
Eiscal  Year  1992  to  Eiscal  Year  1999.  The  updated  study  used  the  term  explicitly 
bundled  contraet  (EBC)  in  place  of  the  CBC  term  in  the  first  study,  but  EBCs  still  reflect 
a  contract  with  dissimilar  SICs,  types  of  contracts,  or  places  of  performanee  (Eagle  Eye 
Publishers,  2000:  5).  This  study  concluded  that  small  businesses  were  being  negatively 
impacted  by  eontract  bundling,  and  calculated  that  “for  every  increase  of  100  bundled 
contracts  there  was  a  decrease  of  106  contracts  to  small  business;  and  for  every  additional 
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$100  awarded  on  bundled  eontraets  there  was  a  decrease  of  $33  to  small  business”  (Eagle 
Eye  Publishers,  2000:  v). 

In  2001,  RAND  produced  a  report  for  the  Air  Eorce  on  how  to  increase  the  benefits 
and  savings  of  contract  bundling  for  service  contracts  while  protecting  small  business. 
The  RAND  study  concluded,  “Bundled  contracts  can  offer  the  possibility  of  significant 
performance  and  savings  benefits  to  the  Air  Eorce”  (Baldwin  and  others,  2002:  61). 
RAND  recommended  that  to  gain  these  benefits  required  improved  Requests  for 
Information  (REls)  by  Air  Eorce  buyers.  The  study  suggested  that  this  could  be 
accomplished  by  centralizing  procurement  activity,  allowing  personnel  to  become  experts 
at  developing  quality  REls.  By  increasing  the  continuity  and  experience  levels  of  buyers, 
the  Air  Eorce  would  be  more  likely  to  bundle  requirements  in  a  way  that  could  gain  the 
benefits  of  bundling  while  still  protecting  small  business  interests.  Ultimately,  RAND 
suggested,  “Market  research  for  bundling  decisions  should  be  integrated  within  the  Air 
Eorce’s  broader  purchasing  and  supply  management  (PSM)  and  performance-based 
services  acquisition  (PBSA)  activities”  (Baldwin  and  others,  2002:  63). 

Eagle  Eye  further  updated  their  research  in  October  of  2002,  extending  the  report  to 
cover  through  fiscal  year  2001.  This  report  argued,  “the  new,  official  federal  bundled 
contract  indicator  .  .  .  masks  the  harm  to  small  business  caused  by  contract  bundling” 
(Eagle  Eye  Publishers,  2002:  5).  According  to  the  Eagle  Eye  study,  the  federal 
definition: 

.  .  .  Eails  to  address  the  phenomenon  of  “accretive  bundling.”  Accretive 
bundling  occurs  when  contracting  officers  add  new  tasks  to  existing  GSA 
Schedule,  Indefinite  Delivery/Indefinite  Quantity  (IDIQ),  Government 
Wide  Acquisition  Contracts  (GWACs),  and  other  multiple  award-type 
contracts.  Accretive  bundling  has  become  the  more  widely  practiced  form 
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of  bundling  since  the  procurement  reforms  of  the  mid-1990s,  and  bundling 
has  risen  signifieantly  since  these  reforms  were  implemented.  (Eagle  Eye 
Publishers,  2002:  6) 

This  report  also  changed  its  definition  slightly,  substituting  Produet  Service  Codes 
(PSCs)  for  SIC  codes.  This  ehange  was  required  beeause  the  government  switched  to 
using  North  Ameriean  Industrial  Codes  (NAICS)  and  stopped  traeking  SICs  in  EY  2001 
(Eagle  Eye  Publishers,  2002:  11).  This  Eagle  Eye  report  reiterated  that  bundling  was 
inereasing  in  federal  proeurement  and  it  was  negatively  impacting  small  business. 

Einally,  the  Offiee  of  Management  and  Budget  prepared  a  strategy  for  dealing  with  the 
effects  of  contract  bundling  on  small  business  in  Oetober  2002.  This  report  used  three 
statisties  to  support  its  statement  that  “the  effeet  of  this  increase  in  contract  bundling  over 
the  past  ten  years  eannot  be  underestimated”  (Office  of  Management  and  Budget,  2002: 
ii).  Eirst,  the  study  found  that  the  number  of  new  definitive  contracts  (over  $25,000)  had 
sharply  declined,  from  over  60,000  in  DoD  in  1990  to  under  20,000  by  2001  (Office  of 
Management  and  Budget,  2002:  4).  Seeond,  the  number  of  small  business  contractors 
receiving  new  contract  award  government  wide  had  decreased  from  over  25,000  in  1990 
to  just  over  10,000  in  2001  (Office  of  Management  and  Budget,  2002:  5).  Einally,  the 
number  of  dollars  for  orders  under  existing  eontracts  increased  from  $21  billion  in  1990 
to  over  $72  billion  in  2001  (Offiee  of  Management  and  Budget,  2002:  6).  The  0MB 
study  agreed  with  the  Eagle  Eye  report  that  the  federal  definition  of  bundled  contract  was 
very  narrow  and  proposed  changes  to  limit  the  impact  of  bundling  on  small  business. 

Summary  of  Contract  Bundling. 

Contract  bundling  has  increased  over  the  past  decade,  in  part  due  to  the  aequisition 
reforms  of  the  1990s.  The  inereased  eontract  bundling  has  negatively  impaeted  small 
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business.  Unfortunately,  varying  definitions  of  a  bundled  eontract  and  limited  data 
eolleetion  have  hampered  efforts  to  quantify  that  negative  impaet. 


Performance-Based  Services  Acquisition  (PBSA) 

PBSA  differs  from  eontract  bundling  in  that  no  studies  have  been  conducted  to  assess 
how  the  implementation  of  PBSA  has  affected  small  business,  as  this  research  will.  The 
purpose  of  this  section  of  the  literature  review  is  to  review  the  implementation  of  PBSA 
in  the  Air  Force  and  the  reasons  for  its  use. 

Definition, 

According  to  the  FAR,  there  are  four  characteristics  of  performance-based  contracts: 

Performance-based  contracts — 

(a)  Describe  the  requirements  in  terms  or  results  required  rather  than  the 
methods  of  performance  of  the  work; 

(b)  Use  measurable  performance  standards  (i.e.,  terms  of  quality, 
timeliness,  quantity,  etc.)  and  quality  assurance  surveillance  plans; 

(c)  Specify  procedures  for  reductions  of  fee  or  for  reductions  to  the  price 
of  a  fixed-price  contract  when  services  are  not  performed  or  do  not 
meet  contract  requirements;  and 

(d)  Include  performance  incentives  where  appropriate.  (Federal 
Acquisition  Regulation,  2003:  37.601) 

The  idea  behind  PBSA  is  to  switch  from  a  prescriptive,  detailed  requirement  to  an  open- 
ended,  results-based  requirement  that  will  allow  a  contractor  to  be  innovative  in  how  the 
requirement  is  met.  Although  this  research  primarily  uses  the  term  PBSA,  it  may  be  used 
interchangeably  with  the  term  performance-based  service  contracts  (PBSC),  which 
appears  in  several  quotations  in  the  following  section. 

Implementation  of  PBSA. 

PBSA  became  federal  policy  in  April  of  1991,  when  the  Office  of  Federal 
Procurement  Policy  (OFPP)  required  agencies  to  use  PBSA  “to  the  maximum  extent 
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practicable  when  acquiring  services”  (Ausink  and  others,  2001:  vii).  As  previously 
mentioned,  the  FAS  A  legislation  of  1994  also  stressed  the  use  of  PBS  A  for  serviees 
aequisition.  The  FAR  was  amended  in  1997  to  reflect  the  new  poliey  and  legislation. 

The  Air  Foree  published  Air  Foree  Instruetion  (AFI)  63-124,  Performance-Based 
Service  Contracts  (PBSC)  on  April  1,  1999  to  “preseribe  how  the  Air  Foree  will 
implement  PBSC  and  quality  assuranee  in  aeeordanee  with  the  FAR”  (Ausink  and  others, 
2001:  vii).  A  year  later,  in  April  2000,  Jaeques  Gansler,  the  Under  Seeretary  of  Defense 
for  Aequisition  and  Teehnology,  issued  a  memorandum  deelaring,  “At  a  minimum,  that 
50  pereent  of  service  aequisitions,  measured  both  in  dollars  and  actions,  are  to  be 
performanee-based  by  the  year  2005”  (Gansler,  2000). 

Purpose  of  PBSA  and  the  Results  of  Implementation. 

The  DoD  Guidebook  for  PBSA  details  five  objeetives  of  using  PBSA:  maximize 
performanee,  maximize  eompetition  and  innovation,  encourage  and  promote  the  use  of 
eommereial  serviees,  shift  in  risk  (from  the  government  to  the  eontraetor),  and  aehieve 
savings  (Offiee  of  the  Under  Seeretary  of  Defense  (Aequisition  Reform),  2000:  5).  In  his 
April  2000  memorandum,  Gansler  states,  “In  order  to  maximize  performanee,  innovation, 
and  eompetition,  often  at  lower  eost,  performanee-based  strategies  for  the  aequisition  of 
services  are  to  be  used  whenever  possible”  (Gansler,  2000). 

However,  a  report  of  the  Interageney  Task  Foree  on  Performanee-Based  Serviee 
Aequisition  published  in  July  2003  questions  the  monetary  savings  of  PBSA,  stating  that, 
“There  is  little  eurrent  data  to  support  monetary  savings,  and  if  sueh  data  did  exist,  it 
would  be  extremely  diffieult  to  isolate  the  exaet  reasons  the  savings  oecurred”  (Offiee  of 
Management  and  Budget,  2003:  10).  A  RAND  report  isolated  three  reasons  why 
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determining  savings  from  PBSA  was  challenging:  contracts  change  in  scope,  government 
cost  estimates  may  be  inaccurate,  and  most  savings  would  be  internal  costs,  and  thus  hard 
to  measure  (Ausink  and  others,  2001:  34).  While  the  report  found  that  government 
personnel’s  opinion  of  PBSA  had  improved  and  that  those  same  personnel  perceived 
improvement  in  contract  performance,  “surprisingly  little  credit  was  given  to  PBSA”  for 
that  improvement  (Ausink  and  others,  2001:  33). 

With  only  two  years  remaining  before  the  requirement  that  50  percent  of  service 
contracts  be  acquired  using  PBSA,  measures  of  how  widespread  it  is  being  used  are  not 
available.  According  to  the  0MB  report,  “Agencies  have  made  moderate  progress  in 
implementing  PBSA,  but  have  experienced  difficulties  in  applying  PBSA  effectively” 
(Office  of  Management  and  Budget,  2003:  foreword). 

Summary  of  Performance-Based  Services  Acquisition. 

Legislation  and  regulation  requires  DoD  and  the  Air  Force  to  acquire  services  using 
PBSA.  Although  the  use  of  PBSA  is  assumed  to  be  increasing,  an  exact  measure  of  its 
use  is  unavailable.  In  addition,  its  effects  on  small  business  have  not  been  determined. 
This  research  will  attempt  to  provide  some  insight  into  that  question. 

Transformation 

The  previous  sections  have  reviewed  prior  acquisition  reform  efforts  and  initiatives. 
This  section  looks  at  current  efforts  to  change  the  way  the  government  procures  materials 
and  services.  Current  procurement  reforms  are  classified  under  the  broader  term 
transformation,  which  applies  to  a  comprehensive  review  of  all  aspects  of  the  Department 
of  Defense. 
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Defined, 


As  with  many  terms  that  become  buzzwords,  the  exact  meaning  of  transformation 
varies  with  each  individual’s  perspective.  It  is  not  surprising,  then,  that  “many  in  the 
acquisition  community  misunderstand  the  intended  meaning  of  transformation” 
(Manchester,  2002;  73).  Edward  C.  “Pete”  Aldridge,  Jr.,  the  former  Under  Secretary  of 
Defense  (Acquisition,  Technology,  and  Logistics),  says  transformation  has  two  parts: 
“Using  things  we  currently  have  in  better,  more  innovative  ways”  and  “the  innovative  use 
of  new  technology  to  achieve  improvement  in  capability”  (Gasiorek-Nelson,  2003:  22). 
The  Chief  of  Staff  of  the  Air  Force,  General  John  P.  Jumper,  stated,  “There  is  a  lot  of  talk 
about  transformation  today,  and  the  power  of  transformation  ...  is  about  relationships.  It 
is  about  the  commitment ...  to  do  things  in  new  and  different  ways”  (Bowman,  2003b: 
Slide  3).  The  Assistant  Secretary  of  the  Air  Force,  Marvin  R.  Sambur  declares  “We  must 
change  the  way  we  do  business  to  deliver  capability  to  the  warfighter  in  a  timely, 
affordable  manner”  (Sambur,  2003:  1). 

Procurement  transformation  is  the  “alignment  of  policies,  processes,  people  and 
technologies  in  support  of  an  overall  procurement  vision”  (Bowman,  2003a:  Slide  24). 
The  motivation  behind  procurement  transformation  is  “the  Air  Force  must  leverage  its 
‘buying  power’  to  meet  the  warfighters’  needs”  (Bowman,  2003a:  Slide  5).  The  key  to 
procurement  transformation  is  that  “transformation  begins  with  changing  behavior  from 
transactional  to  strategic”  (Bowman,  2003a:  Slide  4). 

Foundational  Research. 

Providing  much  of  the  basis  of  procurement  transformation  efforts  is  a  RAND  study 
commissioned  by  the  Air  Force.  The  study  focused  on  how  to  implement  purchasing  and 
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supply  chain  management  (PSCM)  best  praetiees  from  industry  in  the  Air  Foree. 

Beeause  new  PSCM  “praetiees  in  the  eommereial  seetor  have  been  reported  to 
substantially  improve  performance  and  reduce  the  eosts  of  purehased  goods  and  serviees” 
(Moore  and  others,  2002:  1),  the  Air  Foree  eould  benefit  from  implementing  these 
praetiees.  In  addition,  “major  changes  in  the  FAR  enable  all  DoD  eontraeting 
organizations  to  take  advantage  of  many  of  the  best  eommereial  PSCM  praetiees”  (Moore 
and  others,  2002:  19).  The  study  chronicled  the  most  eommon  eommereial  practices  and 
made  reeommendations  on  how  they  eould  be  implemented  in  the  Air  Foree,  eoneluding 
“Soureing  has  strategie  importanee  for  the  Air  Force”  (Moore  and  others,  2002:  98). 
PSCM  praetiees  ean  help  the  Air  Foree  exploit  soureing’s  strategie  value. 

Implementation. 

Building  on  the  RAND  findings,  the  Air  Foree  has  moved  to  implement  some  of  these 
PSCM  initiatives  through  proeurement  transformation.  But  transformation  extends 
beyond  the  Air  Foree.  The  DoD  has  established  Business  Initiative  Couneils  (BICs)  to 
apply  transformation  to  the  aequisition  eommunity.  The  goal  of  the  BICs  is  “to  improve 
the  efficieney  of  Department  of  Defense  business  operations  by  identifying  and 
implementing  business  initiatives  that  ereate  savings  to  be  realloeated  to  higher  priority 
efforts”  (Manchester,  2002:  73). 

Within  the  Air  Foree,  Sambur  deelares  there  to  be  two  goals  of  proeurement 
transformation:  a  deerease  in  aequisition  eyele  time  and  inereased  eredibility  in 
aequisition  exeeution.  Sambur  states,  “All  transformational  initiatives  must  point  baek 
toward  these  two  imperatives”  (Sambur,  2003:  1).  The  implementation  of  proeurement 
transformation  is  proeeeding  on  several  fronts. 
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One  aspect  of  transformation  is  professional  development  and  education.  The  goal 
here  is  “to  develop  mission- focused,  multi-skilled  business  professionals  leveraging 
world-class  processes  and  technology”  (“Procurement  Transformation  Status  Update  20 
March  2003,”  2003:  Slide  2).  Current  training  and  education  requirements  are  being 
reviewed  to  see  if  they  are  consistent  with  new  priorities.  These  new  requirements 
demand  that  while  focusing  “on  the  future,  we  must  transform  our  workforce  to  become 
commodity  strategists,  supplier  relationship  managers,  and  e-Business  experts” 
(“Procurement  Transformation  Status  Update  20  March  2003,”  2003:  Slide  2). 

One  direct  result  of  the  procurement  transformation  is  the  creation  of  commodity 
councils.  Commodity  councils  are  “cross-functional  sourcing  teams  that  use  a  centralized 
strategy  with  a  decentralized  execution”  (Bowman,  2003a:  Slide  9).  There  are  four  goals 
of  the  commodity  council:  eliminate  duplication  of  effort,  minimize  supply  chain  costs 
through  integration/collaboration,  demonstrate  the  power  of  leveraged  purchasing,  and 
meet  small  business  obligations  and  seize  small  business  opportunities  (Bowman,  2003b: 
Slide  14). 

Impact  on  Small  Businesses. 

While  still  a  recent  initiative,  procurement  transformation  will  impact  small  business 
participation  in  Air  Force  contracting.  The  reforms  made  in  the  name  of  transformation, 
however,  are  reflective  of  changes  occurring  in  the  commercial  marketplace.  According 
to  Lauren  McGregor,  manager  of  supplier  diversity  for  United  Technologies  Corp., 
“Supply  bases  are  consolidating,  and  smaller  businesses  are  typically  not  necessarily  in 
the  position  to  handle  a  national  or  global  size  contract  because  of  the  size  of  their 
business”  (Milligan,  1999:  61).  Protecting  small  businesses  and  supplier  diversity 
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requires  “eompanies  to  seek  out  sustainable,  long-term  relationships”  with  suppliers 
(Milligan,  1999:  61). 

The  ehallenge  in  ineorporating  these  new  eorporate  teehniques  eomes  in  proteeting  the 
small  business  interest.  However,  this  ehallenge  is  not  unique  to  government.  In  faet, 
“many  firms  that  are  sueeessfully  implementing  PSCM  praetiees  have  diseovered  ways  to 
reduee  the  number  of  suppliers  they  purehase  from  while  still  providing  opportunities  for 
SDBs  to  benefit  from  these  purehases”  (Moore  and  others,  2002:  1 1 1).  Furthermore, 
“there  are  suffieient  instanees  of  innovative  eommereial  firms  managing  diversity  in  their 
supplier  base  to  lead  us  to  believe  that  the  Air  Foree  need  not  worry  about  PSCM 
praetiees  eliminating  SDBs  from  the  eompetitions”  (Moore  and  others,  2002:  113). 

To  ensure  small  businesses  are  proteeted  will  require  effort  and  vigilanee.  It  does 
appear  to  be  a  priority  of  proeurement  transformation,  as  “eommodity  eouneils  must 
develop  their  proeurement  strategies  with  full  eonsideration  of  small  business  obligations 
and  opportunities”  (Bowman,  2003b:  Slide  18).  It  is  even  possible  that  some  of  the 
reforms,  ineluding  more  eomprehensive  data  eolleetion  and  “better  spend  analysis 
visibility  ean  work  to  the  advantage  of  small  business”  (Bowman,  2003b:  Slide  18). 

Summary  of  Transformation. 

At  this  point  in  time,  it  is  impossible  to  know  how  proeurement  transformation  will  be 
remembered  and  how  it  will  impaet  small  business  partieipation.  The  only  eertainty  is 
that  proteeting  small  businesses  will  require  eontinued  vigilanee  and  effort.  Although 
there  may  be  no  one  agreed-upon  definition  of  transformation,  it  indisputably  provides  a 
eontinuation  of  the  aequisition  reform  efforts  diseussed  previously. 
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Government  Performance  Results  Act  of 1993  (GPRA) 

The  previous  sections  of  this  chapter  have  provided  a  base  of  knowledge  to  address 
the  first  research  question  of  this  thesis:  Have  acquisition  reform  initiatives  negatively 
impacted  small  business  participation  in  Air  Force  contracting?  The  following  sections 
provide  information  needed  to  answer  the  second  research  question:  Are  there  alternative 
measures  to  the  percentage  of  dollars  metric  that  the  Air  Force  Small  Business  office  can 
use  to  evaluate  the  effectiveness  of  their  program?  Performance  measures,  or  metrics,  for 
the  government  must  be  judged  based  on  the  requirements  of  GPRA,  a  law  designed  to 
“improve  the  confidence  of  the  American  people  in  the  capability  of  the  Federal 
Government,  by  systematically  holding  Federal  agencies  accountable  for  achieving 
program  results”  (United  States  Congress,  1993:  Sec.  2  (b)).  This  section  will  review  the 
challenges  and  results  of  GPRA,  while  the  following  section  will  provide  an  outline  of 
characteristics  of  successful  business  metrics. 

History  of  GPRA. 

GPRA  passed  Congress  and  was  signed  into  law  by  President  Clinton  in  1993.  It 
“intended  to  improve  the  efficiency  and  effectiveness  of  federal  programs  by  establishing 
a  system  to  set  goals  for  program  performance  and  to  measure  results”  (General 
Accounting  Office,  1997:  3).  By  requiring  the  creation  of  performance  measurement 
systems  for  federal  agencies,  GPRA  “intended  to  shift  the  focus  of  government 
decisionmaking,  management,  and  accountability  from  activities  and  processes  to  the 
results  and  outcomes  achieved”  (General  Accounting  Office,  2001:  6).  The  focus  in 
federal  programs  shifts  to  results  under  GPRA.  But  GPRA  was  intended  as  a  starting 
point.  John  Mercer,  who  served  as  Counsel  to  the  Senate  Governmental  Affairs 
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Committee  during  the  passage  of  GPRA,  stated  “from  the  beginning,  GPRA  was  intended 
to  point  the  federal  government  in  a  partieular  direetion  toward  a  generally  defined  vision 
of  improved  government  performance”  (Mercer,  2001). 

Improved  government  performance  depended  upon  the  two  core  values  of  GPRA; 
“Transparency  and  accountability  in  federal  agencies  is  key  to  improving  performance, 
particularly  as  measured  by  program  efficiency  and  effectiveness”  (Mercer,  2001).  These 
two  elements  of  transparency  and  accountability  are  the  heart  of  GPRA.  As  one  observer 
noted  “Prior  to  GPRA,  there  was  a  tendency  for  government  to  ‘manage  activities  and 
hope  for  results.’  Post  GPRA,  with  open  disclosure  and  transparency,  it  is  possible  for 
Congress  to  ‘choose  to  do  only  the  programs  that  will  produce  results”  (McTigue,  2001). 
Senator  Fred  Thompson  chaired  the  Senate  Government  Affairs  Committee  and  noted, 
“For  the  first  time.  Congress  will  see  what  results  taxpayer- funded  programs  are 
achieving.  For  the  Results  Act  to  mean  anything,  it  will  be  up  to  Congress  to  hold 
agencies  accountable  for  those  results”  (Laurent,  2000:  78).  GPRA  did  more  than  shift 
the  focus  on  results;  it  tied  budgets  to  results,  which  caused  a  shift  in  focus  of  the 
government  as  budget  management  became  a  planning  and  management  function  under 
GPRA(Melese,  1999:  19). 

GPRA  also  required  agencies  to  define  strategic  objectives.  The  Department  of 
Defense  identified  four:  integrate  GPRA  into  the  Performance-Planning-Budgeting 
System  (PPBS);  make  GPRA  a  meaningful  Secretary  of  Defense  level  report;  develop 
corporate  level  goals  and  corporate  level  performance  measures,  and  enable  subordinate 
units  to  link  to  the  corporate  goals  through  the  use  of  GPRA  tools  (Maroni,  1998:  12). 
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GPRA  intended  to  ehange  the  focus  of  federal  agencies  and  provide  insights  into  how 
agencies  should  develop  metrics.  Implementing  GPRA  as  Congress  intended,  however, 
faces  several  obstacles. 

The  Obstacles  to  Implementing  GPRA, 

Kautz  and  Netting  believe,  “At  least  three  challenges  to  the  GPRA’s  effectiveness 
seem  likely:  bureaucratic  resistance  to  change,  congressional  and  executive  territorialism, 
and  political  currents”  (Kautz  and  Netting,  1997:  367).  GPRA  requires  a  change  in  how 
agencies  operate  and  plan  for  the  future.  The  challenges  involved  extend  beyond  the 
cultural  resistance  to  change.  Setting  strategic  outcome  goals  are  particularly  challenging 
for  government  agencies  because  of  their  missions;  in  the  commercial  world,  metrics  can 
capture  profit  and  loss  measures,  but  in  government,  “since  many  of  its  agencies  are 
mission  oriented,  organizational  goals  are  not  necessarily  captured  as  cost  savings  or 
profits”  (Chmielewski  and  Phillips,  2002:  226).  Determining  strategic  outcomes  can  be 
more  difficult  when  agencies  “seldom  measure  their  success,  or  their  strategic  objectives, 
in  dollars”  (Blackerby,  1994:  20).  Before  the  bureaucracy  can  be  convinced  to  change  to 
a  strategic  outcome  focus,  agencies  must  determine  those  strategic  outcomes. 

The  second  challenge  Kautz  and  Netting  presents  deals  with  the  power  struggle 
between  the  executive  and  legislative  branches.  First,  both  elements  must  agree  on  what 
agencies  should  be  doing,  because  “until  there  is  agreement  on  goals  (results)  and  hard 
political  decisions  on  overlapping  programs,  the  GPRA  is  limited  in  what  it  can 
accomplish”  (Aberbach,  2002b:  20).  When  this  agreement  is  reached,  agencies  will  be 
able  to  establish  measures  and  collect  data,  but  “data  rarely  are  sufficient  by  themselves 
to  drive  policy”  (White,  2001 :  68).  The  President  and  Congress  must  agree  upon  policy 
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before  effeetive  data  eolleetion  ean  begin.  Finally,  to  be  worthwhile,  both  the  President 
and  Congress  “must  demonstrate  that  results  information  will  be  a  major  influenee  in 
future  deeision-making”  (McTigue,  2001).  These  two  branches  of  government  must 
work  together  if  the  goals  of  GPRA  are  to  be  achieved. 

The  final  challenge  mentioned  by  Kautz  and  Netting  is  politics  and  political  currents. 
While  politics  is  a  major  part  of  reaching  agreement  between  Congress  and  the  President, 
it  presents  two  additional  challenges  to  GPRA.  Implementing  GPRA  costs  agencies  time 
and  money,  and  “much  remains  to  be  done  to  ensure  that  performance  measurement  is 
useful  enough  to  justify  the  cost”  (Wholey,  1997:  129).  In  addition  to  monetary 
pressures,  actions  taken  by  one  federal  agency  can  have  unintended  effects  on  other 
organizations.  A  challenge  to  GPRA  is  that  “determining  the  links  between  what  the 
federal  government  does  and  outcomes  turns  out  to  be  quite  difficult  because  of  the  many 
outside  factors  that  influence  whether  or  not  goals  are  achieved”  (Aberbach,  2002a:  63). 

GPRA  continues  to  faces  obstacles  to  successful  implementation  ten  years  after 
passage.  External  obstacles  such  as  politics  add  to  the  internal  challenge  of  the  federal 
government  determining  what  its  strategic  outcomes  should  be.  The  next  section  will 
look  at  the  results  of  ten  years  of  GPRA. 

The  Results  of  the  Results  Act, 

After  ten  years,  the  federal  government  is  still  trying  to  implement  GPRA.  Testifying 
before  Congress,  Christopher  J.  Mihm,  the  Director  of  Strategic  Issues  for  the  General 
Accounting  Office,  believes  “adopting  a  results-orientation  requires  transforming 
organizational  cultures  to  improve  decisionmaking,  maximize  performance,  and  assure 
accountability — it  entails  new  ways  of  thinking  and  doing  business”  (Mihm,  2001). 
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Overcoming  this  challenge  requires  credible  performance  information,  which,  according 
to  Mihm,  “limited  confidence  in  the  credibility  of  performance  information  is  one  of  the 
major  continuing  weaknesses  with  GPRA  implementation”  (Mihm,  2001).  One  of  the 
potential  challenges  highlighted  in  the  previous  section  still  exists  as  “Congress  continues 
to  budget  using  its  own  system,  thereby  making  it  difficult  to  link  performance  data  to 
congressional  budget  decisions”  (Aberbach,  2002a:  62).  A  lack  of  knowledge  plagues 
GPRA  implementation,  as  “too  many  federal  managers  still  believe  that  GPRA  does  not 
apply  to  them  and  their  responsibilities”  (Mercer,  2001). 

Despite  these  difficulties,  “the  act  has  clearly  succeeded  in  many  ways”  (Aberbach, 
2002a:  62).  Success  often  depends  on  the  perception  of  the  agency  manager  who  can 
view  GPRA  as  “a  threat  to  programs  and  a  burden  for  administrators  or  as  an  opportunity 
to  demonstrate  that  programs  improve  clients’  lives”  (Kautz  and  Netting,  1997:  371). 

But  Kautz  and  Netting  (1997:  371)  caution,  “experience  with  other  reforms  indicates  that 
the  mere  connection  of  evaluation  within  budgeting  does  not  in  itself  reform 
government.” 

Addressing  the  issue  of  small  business,  Welch  believes  that  GPRA  must  succeed. 
Welch  argues,  “Establishing  a  sensible  strategic  plan  for  small  business,  linking  that  plan 
to  agency  performance  goals,  and  unleashing  the  flexibility  of  acquisition  reform,  will 
benefit  not  only  small  business  but  also  the  country”  (Welch,  2000:  108). 

Summary  of  Government  Performance  Results  Act  of  1993, 

GPRA  became  law  with  the  intent  of  changing  the  focus  of  government  from 
processes  to  results.  In  the  past  decade  it  has  overcome  some  of  the  challenges  and  met 
with  some  success,  but  still  faces  many  challenges  to  successful  implementation.  GPRA 
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and  its  requirements  must  be  considered  before  developing  metrics  for  any  government 
organization. 

Metrics 

GPRA  provides  direction  on  how  federal  agencies  should  determine  what 
performance  measures  to  use.  Determining  how  well  proposed  metrics  meet  the 
standards  of  GPRA  is  one  way  to  evaluate  the  usefulness  of  a  measurement.  An 
additional  element  to  consider  in  evaluating  proposed  metrics  is  what  the  commercial 
standard  is  for  developing  performance  measures. 

Deflnitions  and  Terms. 

Metrics  can  be  divided  into  two  groups:  primary  and  secondary.  Primary  metrics 
focus  on  overall  organization  performance,  while  the  focus  of  secondary  metrics  is  more 
internal  and  are  used  to  evaluate  different  units  within  the  organization.  The  focus  of  this 
research  is  primary  metrics,  which  “address  the  results  you  intend  to  produce  and  the 
value  you  export  to  others”  (Frost,  2000:  24).  Primary  metrics  can  help  the  organization 
“align  efforts,  manage  who’s  accountable  for  what,  track  progress,  and  report  results” 
(Frost,  2000:  24).  Two  key  elements  of  metrics  are  their  reliability  and  validity;  a 
reliable  metric  “produces  the  same  result  every  time,  given  the  same  circumstances” 
(Frost,  2000:  66),  while  a  valid  metric  “tracks  what  it’s  supposed  to”  (Frost,  2000:  64). 
An  effective  primary  metric  is  both  valid  and  reliable. 

Why  Metrics? 

Metrics  are  “widely  recognized  as  a  powerful  tool  for  strengthening  organizations  and 
communities,  empowering  people,  facilitating  organizational  development  and  capacity 
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building,  and  managing  change”  (Love,  2001;  442).  Metrics  provide  a  focus  for 
organizations  and  their  members;  elearly  stating  goals  through  metries  allows  individuals 
to  focus  their  drive  on  aehieving  those  goals.  In  summation,  “the  bottom  line  for 
organizations  in  all  seetors  is  the  elear  demand  to  measure  outeomes  and  use  both 
quantitative  and  qualitative  data  to  tell  eompelling  ‘performanee  stories’  about  how  well 
their  strategies  have  worked”  (Love,  2001:  438).  Metries  form  the  foundation  of  these 
“performance  stories”. 

Characteristics  of  Effective  Metrics  and  How  They  Relate  to  Strategy, 

Mark  Graham  Brown  provides  eight  eharacteristics  of  what  he  terms  “world-elass 
performanee  measurements”: 

•  Fewer  are  better:  Coneentrate  on  measuring  the  vital  few  key 
variables  rather  than  the  trivial  many. 

•  Measures  should  be  linked  to  the  factors  needed  for  sueeess:  key 
business  drivers. 

•  Measures  should  be  a  mix  of  past,  present,  and  future  to  ensure  that 
the  organization  is  eoncerned  with  all  three  perspeetives. 

•  Measures  should  be  based  around  the  needs  of  eustomers, 
shareholders,  and  other  key  stakeholders. 

•  Measures  should  start  at  the  top  and  flow  down  to  all  levels  of 
employees  in  the  organization. 

•  Multiple  indiees  ean  be  eombined  into  a  single  index  to  give  a 
better  overall  assessment  of  performanee. 

•  Measures  should  be  ehanged  or  at  least  adjusted  as  the 
environment  and  your  strategy  ehanges. 

•  Measures  need  to  have  targets  or  goals  established  that  are  based 
on  researeh  rather  than  arbitrary  numbers.  (Brown,  1996:  3) 


Eaeh  of  these  elements  is  important  to  an  effeetive  me  trio.  The  key  element  to  a  good 
metrio  is  that  it  must  be  linked  to  the  organization’s  strategy,  whioh  is  one  of  the 
requirements  of  GPRA.  Meeting  the  requirements  of  GPRA  meets  the  framework  of  the 
oommeroial  metric  model. 
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Brown  provides  a  model  for  how  metries  link  to  strategy.  He  argues  that  an 
organization  must  first  establish  what  its  mission,  vision,  and  values  are.  From  there,  the 
organization  ean  identify  what  factors  will  most  influence  the  ability  to  successfully 
accomplish  that  mission.  These  factors  should  be  measured  as  metrics.  The  metrics  then 
can  be  used  to  set  goals  and  objectives.  These  goals  and  objectives  require  a  strategy  to 
achieve  them.  Hence,  identifying  relevant  metrics  is  essential  in  the  strategy 
development  process  (Brown,  1996:  11). 

Challenges  to  Metric  Usage, 

Developing  effective  metrics  is  not  without  its  challenges.  Difficulties  include 
“clarifying  the  logic  that  links  program  outputs  with  desired  long  term  outcomes,  and 
devising  processes  for  verifying  and  validating  performance  data”  (Newcomer,  2001: 
337).  Additionally,  “performance  indicators,  especially  in  the  absence  of  extensive  and 
costly  validation  efforts,  tend  to  oversimplify  and  obscure,  and  are  inappropriate  for  high 
stakes  decisions”  (Mark,  2001:  475). 

Identifying  the  links  between  metrics  and  strategy  is  a  challenge.  Incorporating  the 
complex  factors  that  can  affect  performance  can  be  costly  and  time  consuming.  In  the 
end,  “the  problem  of  utilization  of  performance  measures  emerges  as  a  multifaceted  one, 
where  measures  are  often  not  developed,  developed  only  selectively,  or,  most  notably, 
developed  but  not  used  or  selectively  used”  (Julnes  and  Holzer,  2001 :  694).  Metrics  must 
possess  all  of  the  characteristics  identified  previously  to  be  effective,  but  they  also  must 
be  seen  as  useful  by  the  members  of  the  organization  in  order  to  truly  help  drive 
organizational  strategy.  Without  the  support  of  organizational  leaders  and  members, 
metrics  cannot  achieve  their  potential  benefit. 
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Summary  of  Metrics. 

GPRA  provides  guidelines  and  direetions  for  government  agencies  in  developing 
metrics.  Commercial  firms’  experiences  also  provide  assistance  in  developing  effective 
metrics.  In  many  cases,  GPRA  incorporates  the  lessons  learned  from  industry.  The  key 
test  to  determining  the  effectiveness  of  a  metric  is  whether  or  not  it  is  linked  to  an 
organization’s  mission  and  whether  or  not  it  is  being  implemented  in  the  organization’s 
strategy. 

Conclusion 

This  chapter  has  provided  the  background  information  needed  to  answer  the 
research  questions.  It  has  looked  at  the  basics  of  the  small  business  program,  acquisition 
reform,  and  developing  metrics.  This  chapter  provides  a  foundation  for  answering  the 
research  questions,  and  the  following  chapter  provides  the  method  to  answer  those 
questions. 
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III.  Methodology 


The  purpose  of  this  chapter  is  to  examine  the  methodology  used  to  answer  the 
research  questions.  It  begins  with  a  discussion  of  the  data  used  in  the  research  and  the 
advantages  and  disadvantages  of  using  secondary  data,  followed  by  a  detailed  description 
of  modifications  made  to  the  original  data  set.  Next,  the  chapter  will  compare  two 
methodological  approaches  to  answering  the  research  questions,  address  methods  used  in 
previous  research,  and  consider  potential  stumbling  blocks  of  the  chosen  method.  Then 
the  chapter  will  outline  the  selection  of  dependent  and  independent  variables,  construct 
the  model  used  to  answer  the  research  questions,  list  the  hypotheses  to  be  tested  by  the 
model,  followed  by  a  step-by-step  list  of  the  procedures  to  be  used  to  implement  the 
chosen  methodology.  Finally,  a  construct  will  be  developed  to  determine  whether  or  not 
a  given  metric  is  effective. 

The  Data 

The  data  used  for  this  study  are  from  the  JOOl  database,  which  is  the  Air  Force 
component  of  the  Federal  Procurement  Data  System  (FPDS).  The  data  set  used  is  the 
complete  JOOl  database  from  Fiscal  Year  (FY)  1990  to  FY  2003. 

The  data  consists  of  all  Form  DD-350s  (Department  of  Defense  Individual  Contract 
Action  Reports)  completed  during  that  time  period.  DD-350s  are  completed  any  time  an 
obligation  is  made  on  a  contract  of  at  least  $25,000  and  “describe  the  financial, 
competitive,  statutory,  and  other  characteristics  of  the  obligation”  (Eagle  Eye  Publishers, 
2002).  A  separate  DD-350  is  completed  for  initial  contract  award  as  well  as  any 
modification  of  the  contract.  Modifications  can  be  made  for  a  wide  variety  of  reasons. 
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from  adding  money  to  the  contraet  to  exercising  an  option  to  terminating  the  contract. 

The  DD-350  is  only  completed  for  prime  contracts,  those  directly  between  the 
government  and  the  contractor.  Additional  related  work  performed  by  subcontractors  for 
the  prime  contractor  is  not  captured  in  the  DD-350,  or  any  other  tracking  database  for  Air 
Force  contracting  activities  (Eagle  Eye  Publishers,  2002). 

Using  Secondary  Data. 

Data  can  be  either  primary,  meaning  that  it  comes  from  original  sources  and  is 
collected  by  the  researcher  to  answer  a  specific  question,  or  it  can  be  secondary,  meaning 
that  it  has  already  been  collected  for  another  purpose  (Emory  and  Cooper,  1991;  286). 
Secondary  data  “may  be  used  as  the  sole  basis  for  research”  (Emory  and  Cooper,  1991; 
287),  as  is  the  case  with  this  research.  The  most  obvious  advantage  of  using  secondary 
data  is  that  it  can  be  retrieved  more  quickly  and  inexpensively  than  primary  data.  Two 
major  drawbacks  of  secondary  data  are  that  because  the  researcher  does  not  collect  the 
data  by  design,  the  data  may  not  meet  the  researcher’s  needs  and  may  not  be  timely 
(Emory  and  Cooper,  1991;  287).  These  issues  both  affect  this  research  in  that  the 
available  data  restricts  the  selection  of  independent  variables  to  those  that  data  is 
available  for  and  also  restricts  the  timeframe  of  the  research  to  the  last  fourteen  years. 
Additional  difficulties  with  the  data  used  will  be  addressed  in  the  limitations  section  of 
the  final  chapter  of  this  thesis. 

Modifying  the  data. 

The  data  set  includes  all  Air  Eorce  DD350s.  DD350s  are  generated  by  both  new 
contract  awards  and  by  contract  modifications.  Because  this  research  is  focused  directly 
on  how  different  factors  have  affected  small  business  participation,  the  research  is  only 
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concerned  with  new  contract  awards.  Including  modifications  to  existing  eontracts  in  the 
researeh  may  skew  the  results  by  counting  a  firm  that  received  just  one  contract  award 
with  subsequent  modifications  as  having  received  awards  in  multiple  years.  Therefore, 
the  researeher  removed  all  DD350’s  that  are  for  modifications  from  the  studied  data  set. 

Research  Methodology 

In  dealing  with  purely  quantitative  research,  several  possible  methods  of  manipulating 
the  data  are  available  to  the  researeher.  A  causal  regression  model  method  and  a  time 
series  model  method  were  both  considered  for  this  research.  This  section  will  examine 
the  advantages  and  disadvantages  of  eaeh  method,  defend  the  chosen  method,  review 
methods  used  in  similar  researeh,  and  address  potential  pitfalls  in  the  chosen  method. 

Time  Series  Models, 

A  time  series  is  “a  set  of  observations  Xt,  each  one  being  recorded  for  a  specifie  time  f  ’ 
(Brockwell  and  Davis,  1996:  1).  The  time  series  model  is  “construeted  without  drawing 
on  any  theories  coneerning  possible  behavioural  relationships  between  variables” 

(Harvey,  1981:  viii).  These  models  are  “based  on  the  analysis  of  a  chronologieal 
sequence  of  observations  on  a  particular  variable”  (Gaynor  and  Kirkpatrick,  1994:  5). 
Time  series  are  useful  for  predicting  the  performance  of  one  variable  of  interest  based  on 
its  past  performanee. 

Brockwell  and  Davis  provide  three  uses  for  time  series  models:  they  can  provide  a 
eompaet  description  of  the  data;  they  ean  help  separate  (or  filter)  the  noise  from  the 
signals;  and  they  are  useful  for  simulation  studies  (Broekwell  and  Davis,  1996:  6). 

Harvey  contends  that  the  “main  reason  for  modeling  a  time  series  is  to  enable  foreeasts  of 
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future  values  to  be  made  ...  no  attempt  is  made  to  relate  to  other  variables.  The 
movements  in  are  ‘explained’  solely  in  terms  of  its  own  past,  or  by  its  position  in 
relation  to  time”  (Harvey,  1981:  1). 

Time  series  models  are  based  on  “the  assumption  that  one  may  foreeast  the  value  of  an 
item  by  studying  past  movements  of  that  item  over  time”  (Gaynor  and  Kirkpatrick,  1994: 
5).  They  are  “generated  under  the  assumption  of  no  pattern  change”  (Gaynor  and 
Kirkpatrick,  1994:  7).  In  addition  to  these  assumptions,  a  major  limitation  of  time  series 
models  is  that  they  “require  a  lot  of  data  for  fitting  the  points”  (Gottman,  1981:  57).  If 
limited  data  is  available,  then  the  model  provides  “limited  power  to  detect  differences 
between  means.  Of  course,  there  are  those  rare  cases  when,  in  fact,  this  is  a  necessary 
course  of  action.  However,  in  general,  it  is  not  a  wise  course”  (Gottman,  1981:  57). 

Causal  Regression  Models, 

A  causal  regression  model  is  useful  to  “discover  the  form  (mathematical  curve)  of  the 
relationship  between  all  the  variables  and  to  use  it  to  forecast  future  values  of  the  variable 
of  interest  (dependent  variable)”  (Gaynor  and  Kirkpatrick,  1994:  5).  Causal  regression 
models  “assume  that  the  variable  to  be  forecasted  can  be  explained  by  the  behavior  of 
another  variable  or  set  of  variables  (independent  variables)”  (Gaynor  and  Kirkpatrick, 
1994:  5). 

There  are  two  major  shortcomings  to  using  a  causal  regression  model.  First,  it  can 
only  be  used  for  a  limited  timeframe  and  is  not  appropriate  for  long-term  forecasts. 
Second,  it  provides  “no  method  to  account  for  outside  influences  or  changes  that  would 
affect  the  results”  (Gaynor  and  Kirkpatrick,  1994:  6).  It  can  be  used  to  examine  the 


49 


relationships  between  various  faetors,  but  it  is  unlikely  that  all  faetors  that  affeet  the 
relationship  ean  be  identified. 

The  eausal  regression  model  does  have  some  advantages: 

•  Onee  the  ehoiee  of  independent  variable(s)  is  made,  the  foreeasts  are  based  only 
on  their  predetermined  values  and  thus  are  eompletely  objeetive. 

•  There  are  ways  to  measure  the  aeeuraey  of  the  foreeast. 

•  Onee  the  models  are  eonstrueted,  it  is  less  time-eonsuming  to  generate  foreeasts 
from  them. 

•  They  have  a  means  of  foreeasting  point  estimates  (one  speeifie  value)  or  interval 
estimates  (a  range  of  values  based  upon  a  eonfidenee  interval).  (Gaynor  and 
Fitzpatriek,  1994:  5) 

Finally,  the  eausal  regression  model  is  “generated  based  upon  a  set  of  underlying 
assumptions  eoneerning  both  struetural  and  behavioral  relationships  that  are  assumed  to 
be  the  same  over  the  time  period”  (Gaynor  and  Kirkpatriek,  1994:  7).  Beeause  of  these 
assumptions,  “the  model  must  be  generated  with  great  eare,  eonstantly  mindful  of  the 
ehanges  that  have  oeeurred”  (Gaynor  and  Kirkpatriek,  1994:  7). 

The  Chosen  Methodology. 

Understanding  the  relationship  between  aequisition  reform  initiatives  and  small 
business  partieipation  in  the  Air  Foree  is  the  foeus  of  this  researeh.  Beeause  the  goal  is  to 
understand  the  relationship  between  variables  (in  this  ease,  small  business  eontraetors  and 
variables  related  to  reform  initiatives),  a  eausal  regression  model  approaeh  is  used. 
Additionally,  only  fourteen  data  points  are  available  (one  for  eaeh  year  1990-2003). 
Having  sueh  a  limited  number  of  data  points  in  a  time  series  models  eould  result  in  the 
limited  usefulness  that  Gottman  eautions  against.  Finally,  although  a  eausal  regression 
model  ean  be  used  in  a  predietive  way,  this  researeh  is  not  foeused  on  predieting  future 
events;  rather  it  is  interested  in  attempting  to  understand  past  relationships. 
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Past  Causal  Regression  Models  in  Research, 

Mayer  (1992)  demonstrated  through  use  of  a  eausal  regression  model  that  there  are 
eeonomic  and  electoral  cycles  in  defense  contract  awards.  Mayer’s  variable  of  interest 
was  contract  awards.  His  research  concluded  “defense  contracting  activity  increased 
immediately  before  elections,  both  to  stimulate  the  economy  and  to  advance  the  interests 
of  incumbent  legislators”  (Mayer,  1992:  28).  In  addition,  his  research  found  that  “before 
major  social  programs  were  enacted,  defense  contracts  were  also  used  to  combat 
unemployment”  (Mayer,  1992:  28).  While  the  results  of  Mayer’s  research  may  or  may 
not  be  perceived  as  surprising,  his  research  did  demonstrate  the  effectiveness  of  a  causal 
regression  model  that  incorporates  both  quantitative  and  binary  factors.  For  Mayer,  a  few 
of  his  quantitative  factors  were  non-contract  award  dollars,  total  contract  award  dollars, 
and  the  unemployment  rate.  His  binary  factors  included  whether  or  not  the  United  States 
was  at  war,  whether  it  was  the  last  month  of  the  fiscal  year,  and  whether  or  not  it  was  an 
election  year. 

Barben  (1998)  constructed  a  similar  model  to  measure  the  effects  of  acquisition 
reform  initiatives  as  well  as  political-fiscal  variables  on  the  number  of  protests  of  Air 
Force  contract  awards  made  to  the  General  Accounting  Office  (GAO).  Barben’s  variable 
of  interest  was  the  number  of  Air  Force  GAO  protests  processed  during  each  year.  His 
sole  binary  factor  considered  acquisition  reform  initiatives,  while  his  quantitative  factors 
included  number  of  Air  Force  contract  actions,  total  Air  Force  contract  dollars,  GAO 
protest  sustainment  rate,  as  well  as  the  unemployment  rate  (Barben,  1999:  37). 

While  these  two  models  do  not  specifically  address  the  issues  that  are  the  focus  of 
this  research,  they  do  establish  that  a  causal  regression  model  can  be  used  to  answer 
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questions  similar  to  the  questions  examined  in  this  researeh.  Furthermore,  they  both 
demonstrate  the  effeetiveness  of  eausal  regression  when  eonfronted  with  both 
quantitative  and  binary  faetors. 

Pitfalls  of  the  Causal  Regression  Model, 

There  are  four  major  threats  to  the  validity  of  a  eausal  regression  model.  The  first 
pitfall  is  parameter  estimability.  A  model  requires  suffieient  data  points  to  allow  an 
estimate  of  the  parameters  of  the  model  to  be  made.  This  problem  is  resolved  by  the  rule 
“in  general,  the  number  of  levels  of  observed  x  values  must  be  one  more  than  the  order  of 
the  polynomial  in  x  that  you  want  to  fit”  (McClave  and  others,  1998:  551).  For  example, 
a  seeond  order  polynomial  requires  three  observed  data  points  in  order  to  avoid  the  pitfall 
of  parameter  estimability. 

The  seeond  pitfall  is  multieollinearity.  This  oeeurs  when  two  of  the  independent 
variables  in  the  researeh  are  eorrelated.  Failure  to  identify  this  eorrelation  ean  skew  the 
results  for  the  dependent  variable.  In  dealing  with  multieollinearity,  it  is  useful  to  use  a 
stepwise  regression,  where  eaeh  variable  is  introdueed  independently  into  the  model.  It  is 
important  to  note,  however,  that  even  if  two  independent  variables  are  eorrelated,  eaeh 
may  still  independently  eontribute  information  for  understanding  the  dependent  variable 
(McClave  and  others,  1998:  552). 

The  third  pitfall  occurs  when  a  causal  regression  model  is  used  to  predict  outside  the 
experimental  region.  The  causal  regression  model  reflects  relationships  that  occur  in  a 
certain  timeframe  and  “use  of  the  model  outside  that  range  is  a  dangerous  practice” 
(McClave  and  others,  1998:  552). 
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Finally,  a  pitfall  occurs  with  correlated  errors.  Beeause  “the  values  of  both  the 
independent  and  dependent  variables  are  observed  sequentially  over  a  period  of  time,  the 
observations  tend  to  be  eorrelated  over  time,  whieh  in  turn  often  eauses  the  predietion 
errors  of  the  regression  model  to  be  eorrelated”  (MeClave  and  others,  1998;  551). 
MeClave  suggests  that  to  eombat  this  issue,  a  time  series  model  should  be  eonsidered. 

This  researeh  will  attempt  to  avoid  eaeh  of  these  pitfalls.  First,  all  of  the  variables 
tested  have  at  least  three  data  points,  with  most  having  fourteen.  This  meets  the 
requirement  to  avoid  parameter  inestimability.  Seeond,  stepwise  regression  will  be 
performed  for  the  model  to  identify  multieollinearity  issues.  Third,  the  model  is  not 
being  used  to  prediet  future  events,  but  instead  foeuses  solely  on  the  time  frame  studied. 
Finally,  past  researeh  demonstrates  that  the  eausal  regression  model  ean  avoid  the  pitfall 
of  eorrelated  errors  even  when  the  data  is  of  time  series  form.  Despite  the  potential  for 
problems,  the  eausal  regression  model  is  the  best  available  to  answer  the  researeh 
questions  of  this  thesis. 

Building  the  Model 

Model  building  provides  “a  model  that  will  provide  a  good  fit  to  a  set  of  data  and  that 
will  give  good  estimates  of  the  mean  value  of  y  and  good  predietions  of  future  values  of  y 
for  given  values  of  the  independent  variable”  (Mendenhall  and  Sineich,  1996;  274).  The 
first  step  in  model  building  is  to  identify  the  dependent  variables  and  independent 
variables  that  will  be  used  to  answer  the  researeh  question.  The  model  is  then 
eonstrueted  using  those  variables. 


53 


Dependent  Variables, 

The  dependent  variable,  y,  is  the  response  variable  or  the  variable  to  be  modeled 
(McClave  and  others,  1998;  432).  It  is  the  variable  of  interest  in  the  researeh.  In  this 
researeh,  there  are  two  separate  dependent  variables.  The  first  one  is  the  total  number  of 
small  businesses  receiving  Air  Force  contract  awards  each  year.  The  second  dependent 
variable  is  the  total  number  of  new  small  businesses  receiving  Air  Force  contract  awards 
each  year.  With  two  dependent  variables,  a  model  will  be  constructed  for  each  one. 

Independent  Variables. 

The  independent  variable(s),  x,  are  also  called  the  predictor  variables  as  they  are  used 
to  predict  y  (McClave  and  others,  1998:  432).  The  independent  variables  can  be  either 
quantitative  or  binary  (Mendenhall  and  Sincich,  1996:  275).  A  quantitative  independent 
variable  “is  one  that  assumes  numerical  values  corresponding  to  the  points  on  a  line” 
(Mendenhall  and  Sincich,  1996:  275).  The  model  will  incorporate  six  quantitative 
independent  variables:  number  of  Air  Force  contract  actions,  amount  of  Air  Force 
contract  dollars,  number  of  performance-based  service  contracts,  number  of  bundled 
contracts,  number  of  government  wide  agency  contracts,  and  the  number  of  multiple 
award  contracts.  A  binary  independent  variable  is  not  associated  with  a  numerical  scale, 
so  it  requires  the  use  of  dummy  (indicator)  variables  to  make  the  model  work 
(Mendenhall  and  Sincich,  1996:  303).  In  addition  to  the  six  quantitative  variables  above, 
the  model  includes  two  binary  independent  variables:  the  first  variable  is  whether  or  not 
the  Federal  Acquisition  Streamlining  Act  was  in  effect,  and  the  second  variable  is 
whether  or  not  the  Clinger-Cohen  Act  was  in  effect.  These  eight  variables  were  chosen 
carefully  based  on  the  results  of  the  literature  review  as  well  as  the  focus  of  the  study. 
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The  Model, 


Because  there  are  two  dependent  variables  of  interest,  there  are  two  models  used  in  the 
research.  The  two  models,  however,  share  the  same  independent  variables  and  are  thus, 
identical  in  equation  form.  The  model  is: 

y  =  Po  +  Pi  (xi)  +  Pi  (xi)  +  Ps  (xs)  +  P4  (xp  +P5  (xs)  +  P6  (X6)  +  Pi  (xi)  +  Ps  (xs)  +  ^ 

where 

y  =  Number  of  Air  Force  small  business  contractors  in  model  1  and  Number  of  new  Air 

Force  small  business  contractors  in  model  2 

Xi  =  Air  Force  contract  actions 

X2  =  Air  Force  contract  dollars 

X3  =  Number  of  performance  based  service  contracts 

X4  =  Number  of  bundled  contracts 

X5  =  Number  of  government  wide  agency  contracts 

X6  =  Number  of  multiple  award  contracts 

xy  =  Federal  Acquisition  Streamlining  Act:  0  if  prior  to  FASA,  1  if  after  FASA 
Xs  =  Clinger-Cohen  Act:  0  if  prior  to  Cbnger-Cohen,  1  if  after  Clinger-Cohen 
e  =  Random  error  component 

Hypotheses  to  be  Tested 

The  causal  regression  model  provides  an  indication  of  the  relationship  between  the 
dependent  and  independent  variables.  It  cannot,  however,  prove  cause  and  effect  in  those 
relationships  because  the  independent  variables  are  observational  in  nature  as  opposed  to 
experimental  (Mendenhall  and  Sincich,  1996:  349).  The  research  uses  observational  data 
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so  a  statistically  significant  relationship  between  y  andx,  “simply  means  thatx 
eontributes  information  for  the  prediction  ofy”  (Mendenhall  and  Sineieh,  1996:  348). 

Despite  this  restrietion,  the  model  will  allow  various  hypotheses  to  be  tested  that  ean 
provide  insight  into  the  answers  to  the  researeh  questions.  Following  are  the  sixteen 
hypotheses  to  be  tested  in  the  research. 

Barben’s  research  linked  an  inereased  number  of  eontraet  aetions  to  an  inereased 
number  of  eontraetor  protests.  The  first  two  hypotheses  suggest  that  a  similar 
relationship  holds  between  an  inereased  number  of  eontraet  aetions  and  the  number  of 
small  business  eontraetors.  One  assumption  made  in  these  hypotheses  is  that  additional 
awards  are  going  to  additional  eontraetors,  not  just  the  same  eontraetors  getting  multiple 
awards. 

H^:  An  inerease  in  Air  Foree  eontraet  aetions  will  inerease  the  number  of 
Air  Force  small  business  eontraetors. 

H?:  An  inerease  in  Air  Foree  eontraet  aetions  will  inerease  the  number  of 
new  Air  Force  small  business  contractors. 

Barben  also  linked  an  inerease  in  Air  Foree  dollars  spent  to  an  inerease  in  number  of 
eontraetor  protests.  Hypotheses  3  and  4  suggest  a  similar  relationship  between  inereased 
Air  Foree  dollars  and  number  of  small  business  eontraetors.  One  assumption  made  here 
is  that  the  additional  money  is  assoeiated  with  additional  awards  and  additional 
eontraetors. 

H3:  An  increase  in  Air  Force  dollars  spent  will  inerease  the  number  of  Air 
Foree  small  business  eontraetors. 

H4:  An  inerease  in  Air  Foree  dollars  spent  will  inerease  the  number  of  new 
Air  Force  small  business  contractors. 
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Hypotheses  5  and  6  are  the  first  to  test  the  effeets  of  the  aequisition  reforms 
eonsidered  in  this  researeh.  No  previous  researeh  has  doeumented  any  relationship 
between  performanee-based  serviee  contracts  and  small  business  participation. 

Therefore,  the  hypotheses  test  a  negative  relationship  in  order  to  identify  any  negative 
impacts  of  these  reforms. 

H5:  An  increase  in  performance-based  service  contracts  will  decrease  the 
number  of  Air  Force  small  business  contractors. 

He:  An  increase  in  performance-based  service  contracts  will  decrease  the 
number  of  new  Air  Force  small  business  contractors. 

Several  studies,  including  those  by  the  Eagle  Eye  publishers  and  the  General 
Accounting  Office,  concluded  that  bundled  contracts  have  greatly  reduced  the  number  of 
small  businesses  receiving  federal  contracts.  Hypotheses  7  and  8  will  see  if  those 
findings  hold  when  looking  only  at  Air  Eorce  contracts.  It  is  important  to  note  that  the 
definition  of  bundling  used  in  this  research  reflects  the  EAR  definition,  as  that  is  the  one 
used  on  the  DD350  form  that  serves  as  the  basis  of  the  data  used  in  the  research. 

H7:  An  increase  in  bundled  contracts  will  decrease  the  number  of  Air 
Eorce  small  business  contractors. 

Hg:  An  increase  in  bundled  contracts  will  decrease  the  number  of  new  Air 
Eorce  small  business  contractors. 

The  literature  review  raised  the  idea  that  small  businesses  are  less  likely  to  receive 
awards  if  the  contracts  are  very  large  in  dollars  or  complex  in  scope.  GWACs  tend  to 
meet  both  of  these  criteria.  Hypotheses  9  and  10  test  whether  an  increase  in  the  number 
of  actions  issued  against  GWACs  leads  to  a  decrease  in  the  number  of  Air  Eorce  small 
business  contractors. 
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Hg:  An  increase  in  the  number  of  government  wide  ageney  eontraet 
aetions  will  deerease  the  number  of  Air  Foree  small  business 
eontraetors. 

An  inerease  in  the  number  of  government  wide  ageney  eontraet 
aetions  will  deerease  the  number  of  new  Air  Foree  small  business 
eontraetors. 

Similarly  to  the  GWACs,  MACs  will  negatively  affeet  small  businesses  for  the  same 
two  reasons.  First,  they  tend  to  be  larger,  more  eomplex  eontraets.  Seeond,  the  use  of 
MACs  would  tend  to  deerease  the  number  of  new  eontraet  awards.  Hypotheses  1 1  and 
12  test  whether  an  inereased  use  of  MACs  hurts  Air  Foree  small  business  partieipation. 

Hn:  An  inerease  in  the  number  of  multiple  award  eontraet  aetions  will 
deerease  the  number  of  Air  Foree  small  business  eontraetors. 

H^:  An  inerease  in  the  number  of  multiple  award  eontraet  aetions  will 
deerease  the  number  of  new  Air  Foree  small  business  eontraetors. 

Hypotheses  13,  14,  15,  and  16  speak  direetly  to  the  heart  of  the  researeh  question. 
Senator  Kerry  and  Congresswoman  Velazquez  have  eaeh  suggested  that  the  aequisition 
reforms  of  the  1990s  have  negatively  impaeted  small  business  partieipation.  The  reports 
of  the  Eagle  Eye  publishers  and  the  General  Aeeounting  Offiee  provide  data  to  support 
those  suggestions.  Hypotheses  13  and  14  test  whether  EASA  hurt  Air  Eoree  small 
business  partieipation.  Hypotheses  15  and  16  test  whether  Clinger-Cohen  hurt  Air  Eoree 
small  business  partieipation. 

Hn:  EASA  led  to  a  deerease  in  the  number  of  Air  Eoree  small  business 
eontraetors. 

H14:  EASA  led  to  a  deerease  in  the  number  of  new  Air  Eoree  small 
business  eontraetors. 

Hn:  Clinger-Cohen  led  to  a  deerease  in  the  number  of  Air  Eoree  small 
business  eontraetors. 
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H^:  Clinger-Cohen  led  to  a  decrease  in  the  number  of  new  Air  Force 
small  business  contractors. 

These  sixteen  hypotheses  suggest  factors  that  have  influenced  small  business 
participation.  Figure  1  pictorially  demonstrates  the  relationship  between  the  eight 
independent  variables,  the  two  dependent  variables,  and  the  sixteen  hypotheses. 


Independent  Variables  Dependent  Variables 


Figure  1.  Relationship  of  Independent  and  Dependent  Variables  to  Hypotheses 
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Step-by-Step  Procedures 

The  following  steps  will  be  used  to  implement  this  methodology: 

1)  Data  on  Air  Foree  contracting  actions  from  1990-2003  will  be 
obtained  from  the  JOOl  database.  Actions  that  are  modifications 
will  be  removed  from  the  data  as  previously  discussed.  Totals  for 
each  of  the  dependent  and  independent  variables  will  then  be 
calculated  and  listed  in  tabular  form. 

2)  The  dependent  variables  will  be  graphed  to  visually  distinguish 
any  long-term  trends.  Descriptive  statistics  will  be  used  to  analyze 
the  normality  of  the  dependent  variables. 

3)  Each  of  the  dependent  variable  to  independent  variable 
relationships  will  be  plotted  in  a  scattergram  format  to  analyze  the 
possible  relationship.  The  scattergram  may  reveal  whether  the 
relationship  is  positive,  negative,  non-existent,  linear,  or  quadratic. 

4)  The  complete  multiple  regression  model  developed  in  this  chapter 
will  then  be  tested  using  the  JMP  statistical  software  program. 

This  test  will  estimate  the  model  coefficients,  allowing  a  least 
squares  model  to  be  built.  The  complete  model  will  be  examined 
for  usefulness  by  evaluating  the  R-squared  score.  Each  variable 
will  then  be  tested  using  the  t-statistic  test.  These  tests  will 
identify  if  the  model  as  a  whole  is  useful  and  identify  which 
independent  variables  should  be  included  in  the  final  analysis. 

5)  The  new  model  incorporating  only  the  relevant  independent 
variables  will  be  tested  using  the  multiple  regression  features  of  the 
JMP  statistical  software.  This  test  will  determine  the  usefulness  of 
the  new  model. 

6)  The  usefulness  of  the  new  model  will  be  compared  to  the  original 
model  to  ensure  the  proper  independent  variables  remain  in  the 
model. 

7)  Based  on  the  results  of  the  previous  six  steps,  the  hypotheses  will 
be  tested. 


Determining  the  Effectiveness  of  a  Metric 

This  section  consists  of  the  methodology  used  to  answer  the  second  research  question: 
Should  the  Air  Eorce  Small  Business  office  incorporate  the  measures  of  total  small 
business  contractors  and  total  new  small  business  contractors  into  their  metrics?  The 
literature  sets  standards  for  determining  the  effectiveness  of  metrics.  Eove  (2000)  points 
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out  that  a  metric  must  be  measurable  and  quantifiable  to  be  of  value.  Brown  (1996),  in 
his  model,  defines  effective  metrics  as  those  that  are  consistent  with  organizational  goals, 
allow  for  the  creation  of  research-based  goals,  and  promote  the  development  of  strategy 
to  achieve  those  goals.  Julnes  and  Holzer  (2001)  emphasize  the  importance  of  support 
from  organizational  leaders  to  promote  organizational  acceptance.  Based  on  these 
concepts,  the  following  questions  will  be  used  to  test  the  effectiveness  of  the  proposed 
metrics: 

1)  Is  the  measurement  quantifiable? 

2)  Is  the  metric  consistent  with  the  goals  of  the  organization? 

3)  Can  the  metric  establish  specific,  research-based  targets  for  the  organization? 

4)  Can  a  strategy  be  developed  to  meet  the  targets  of  the  metric? 

5)  Does  the  metric  have  the  support  of  organizational  leadership? 

By  testing  the  proposed  metrics  with  these  five  questions,  this  research  will  determine 
the  appropriateness  of  the  Air  Force  Small  Business  office  using  these  two  measures.  In 
determining  the  appropriateness  of  the  metrics,  the  second  research  question  will  be 
answered. 

Conclusion 

This  chapter  has  provided  a  review  of  the  methodology  used  in  the  research.  It  has 
covered  the  data  set  used,  the  type  of  model  chosen,  the  development  of  the  model,  and 
identified  the  hypotheses  to  be  tested  by  the  model.  It  has  provided  a  step-by-step  listing 
of  how  the  hypotheses  will  be  tested.  It  has  also  created  a  construct  of  five  questions  that 
can  determine  the  appropriateness  of  proposed  metrics.  Chapter  IV  follows  both  the  step- 
by-step  procedures  and  metrics  construct.  Chapter  V  then  applies  the  results  to  the 
research  questions  forwarded  from  Chapter  I. 
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Chapter  IV:  Data  Analysis  and  Results 


The  purpose  of  this  chapter  is  to  implement  the  methodology  presented  in  Chapter  III. 
This  chapter  presents  the  results  of  the  step-by-step  procedures  and  tests  the  hypotheses 
developed  in  Chapter  III.  This  chapter  also  provides  analysis  of  the  metrics  construct 
developed  in  Chapter  III. 

Step  1:  Collection  and  Compilation  of  the  Data 

The  researcher  obtained  data  for  all  contracting  actions  recorded  in  the  JOOl  database 
for  the  period  covering  Fiscal  Year  1990  to  Fiscal  Year  2003.  The  data  was  then 
reduced,  in  accordance  with  the  discussion  in  Chapter  III,  to  remove  all  modifications 
from  the  data.  This  was  done  by  removing  all  actions  that  contained  a  value  in  Block 
B13D,  the  Modification  block,  of  the  DD  Form  350.  The  remaining  data  set  will  be 
referred  to  as  the  primary  data  set  in  this  research. 

Each  fiscal  year’s  data  was  then  measured  for  the  variables  involved  in  the  research. 
First,  the  total  number  of  contract  actions  and  total  dollars  awarded  were  calculated  based 
on  the  total  set  of  primary  data.  Next,  the  complete  primary  data  set  was  used  to 
determine  the  number  of  government-wide  agency  contracts  (GWACs)  and  multiple- 
award  contracts  (MACs)  for  each  fiscal  year.  The  number  of  GWACs  was  determined 
using  the  B13A  block  of  the  DD  Form  350.  Any  action  coded  as  an  order  under  a  federal 
schedule  or  as  a  BPA  (Blanket  Purchase  Agreement)  order  under  a  federal  schedule  was 
counted  as  a  GW  AC.  In  addition,  if  block  BI3F  of  the  DD  Form  350  was  coded  as  being 
Government-wide,  then  that  action  was  also  counted  as  a  GWAC.  Actions  were  counted 
as  MACs  if  block  B13C  of  the  DD  Form  350  was  coded  “M”  for  multiple  award.  It  is 
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significant  to  note  that  the  first  year  that  MACs  were  reeorded  in  the  DD350  was  FY 
1996.  This  limitation  required  adjustments  within  the  model  testing  proeess. 

Of  greater  coneern,  however,  was  the  measurement  of  bundled  contraets  and  PBSCs. 
Bloek  BIC  of  the  DD  Form  350  reeords  whether  or  not  a  eontraet  is  bundled.  Bloek  BIE 
reeords  whether  the  eontraet  is  PBSC  or  not.  Unfortunately,  eaeh  of  these  statistics  has 
only  been  reported  in  the  DD350  sinee  FY  2001.  Although  the  totals  are  ineluded  in 
Table  1,  they  eannot  be  used  in  the  regression  model  beeause  of  the  limited  (three) 
number  of  data  points  for  eaeh  measure. 

After  all  of  the  independent  variables  were  reeorded,  the  primary  data  set  was  redueed 
again  in  order  to  measure  the  dependent  variables.  All  aetions  not  eoded  as  being  a 
“Small  Disadvantaged  Business  (SDB)  Performing  in  the  U.S.”  or  as  an  “Other  Small 
Business  (SB)  Performing  in  the  U.S.”  in  bloek  DIA  of  the  DD  Form  350  were  removed 
from  the  primary  data  set.  The  remaining  aetions,  refleeting  all  awards  to  small  business, 
were  sorted  by  Bloek  B5A  of  the  DD  Form  350,  the  eontractor  identifieation  number 
(DUNS,  or  Dun  &  Bradstreet)  bloek.  All  non-unique  DUNS  numbers  were  removed, 
providing  a  set  that  included  only  each  small  business  awarded  a  eontraet.  This  count 
provided  the  number  for  the  total  number  of  small  businesses  dependent  variable. 

The  next  step  required  eompiling  all  the  unique  small  businesses  for  all  fiseal  years 
into  a  data  set  and  sorting  by  fiseal  year  of  award.  Then,  all  non-unique  DUNS  were 
onee  again  removed  from  the  data  set,  providing  a  list  of  all  small  businesses  awarded 
eontraets  sorted  by  when  eaeh  business  first  reeeived  a  eontraet.  FY  1990  was  used  as 
the  base  year,  and  a  eount  was  done  for  eaeh  following  year  of  the  number  of  new  small 
businesses.  An  example  provides  insight  into  the  usefulness  of  this  approaeh.  Assume 
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that  Small  Business  X  has  won  four  Air  Force  contract  awards,  one  in  FY  1992,  two  in 
FY  1997,  and  one  in  FY  2002.  Small  Business  X  will  count  as  one  small  business  for 
each  of  these  three  years,  but  will  only  count  as  a  new  small  business  in  FY  1992. 

The  remaining  two  independent  variables,  FASA  and  Clinger-Cohen,  are  binary,  and 
coded  according  to  the  procedures  outlined  in  Chapter  III.  The  results  of  this  data 
analysis  are  compiled  in  Table  1. 


Table  1.  Summary  of  Dependent  and  Independent  Variables 


Fiscal 

Year 

#Of 

Small 

Businesses 

#  Of  new 
Small 

Businesses 

Total 

Contract 

Awards 

Total 

Dollars 

Awarded 

(OOO’s) 

GWACS 

MACS 

#Of 

Bundled 

Contracts 

#Of 

PBSCs 

FASA 

Clinger- 

Cohen 

1990 

3908 

X 

30,800 

$12,018,113 

4493 

X 

X 

X 

0 

0 

1991 

3804 

2206 

32,156 

$16,099,119 

4525 

X 

X 

X 

0 

0 

1992 

3733 

1959 

28,511 

$9,748,949 

4554 

X 

X 

X 

0 

0 

1993 

4770 

4142 

33,870 

$14,421,807 

5231 

X 

X 

X 

0 

0 

1994 

4391 

2558 

35,142 

$14,280,275 

5310 

X 

X 

X 

0 

0 

1995 

4651 

2253 

37,807 

$13,303,025 

7139 

X 

X 

X 

1 

0 

1996 

5484 

2712 

37,443 

$16,586,586 

5845 

9434 

X 

X 

1 

0 

1997 

5249 

3772 

35,352 

$15,082,151 

5024 

4855 

X 

X 

1 

1 

1998 

4829 

2390 

34,751 

$14,002,955 

5848 

3339 

X 

X 

1 

1 

1999 

4655 

1989 

35,559 

$13,018,073 

7092 

2956 

X 

X 

1 

1 

2000 

4633 

1854 

36,573 

$19,897,905 

6841 

2862 

X 

X 

1 

1 

2001 

4276 

1654 

33,930 

$19,813,732 

6938 

2690 

55 

2555 

1 

1 

2002 

4745 

1885 

39,887 

$17,823,747 

9089 

3553 

41 

4112 

1 

1 

2003 

5979 

2631 

49,172 

$23,551,210 

11470 

4727 

20 

6227 

1 

1 

Step  2:  Analysis  of  the  Dependent  Variables 

Figure  2  provides  a  graph  of  the  dependent  variables.  The  number  of  small  business 
contractors  appears  to  be  in  a  general  upward  trend,  with  a  particularly  sharp  spike  in  FY 
2003.  By  consulting  the  information  in  Table  1,  it  is  noted  that  FY  2003  is  also  the  peak 
year  for  total  contracts  awarded  and  total  dollars  awarded.  In  contrast,  the  number  of 
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new  small  businesses  is  relatively  stable  throughout  the  process  except  for  two  spikes  in 
FY  1993  and  FY  1997.  None  of  the  information  in  Table  1  relates  a  clear  reason  for 
these  spikes. 


Figure  2,  Graph  of  Dependent  Variables 


Table  2  contains  various  descriptive  statistics  regarding  the  two  dependent  variables. 
These  statistics  are  useful  for  further  analysis  of  the  variables.  The  mean  number  of 
small  businesses  contracting  with  the  Air  Force  from  FY  1990  to  FY  2003  is  4650.5,  with 
a  standard  deviation  of  63 1 .9.  The  mean  number  of  new  small  businesses  contracting 
with  the  Air  Force  from  FY  1991  to  FY  2003  is  2461.9,  with  a  standard  deviation  of 
740.8.  Only  the  sharp  increase  in  FY  2003  of  small  business  contractors  falls  more  than 
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two  standard  deviations  from  the  mean.  Similarly,  only  the  spike  of  FY  1993  falls  more 
than  two  standard  deviations  from  the  mean  of  new  small  business  eontraetors. 


Table  2,  Descriptive  Statistics  for  the  Dependent  Variables 


Nnmber  of  Small 

Bnsinesses 

Nnmber  of  New  Small 

Bnsinesses 

Mean 

4650.5 

2461.9 

Standard  Error 

168.9 

205.5 

Standard  Deviation 

631.9 

740.8 

Maximnm 

5979 

4142 

Minimnm 

3733 

1654 

Range 

2246 

2488 

Connt 

14 

13 

Wilk-Shapiro  W 

Test  for  Normality 

0.950799 

0.843688 

The  Wilk-Shapiro  W  Test  for  Normality  feature  of  the  JMP  statistieal  software 
paekage  reveals  that  the  number  of  small  businesses  follows  an  approximately  normal 
distribution,  as  the  .95  seore  indieates  a  failure  to  rejeet  the  null  hypothesis  of  normality 
at  the  .05  signifieanee  level.  The  number  of  new  small  businesses  seore  of  0.84, 
however,  results  in  a  rejeetion  of  the  null  hypothesis  of  normality  at  the  .05  signifieanee 
level.  The  number  of  new  small  businesses  does  not  follow  an  approximately  normal 
distribution.  This  is  further  supported  by  the  high  standard  deviation  and  that  the  range 
of  the  numbers  is  greater  than  the  mean.  However,  “it  is  not  necessary  that  data  follow  a 
normal  distribution  for  multiple  linear  regression  to  be  valid”  (Dallal,  2002). 
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Step  3:  Plotting  the  Relationships  Between  the  Dependent  and  Independent  Variables 

A  scattergram  can  be  used  to  determine  a  possible  relationship  between  two  variables. 
If  the  plotted  points  follow  a  generally  upward-pointing  line,  then  the  two  variables  may 
share  a  positive  relationship.  If  the  plotted  points  follow  a  generally  downward-pointing 
line,  then  the  two  variables  may  share  a  negative  relationship.  If  the  plotted  points  appear 
to  be  randomly  distributed,  then  no  relationship  is  suggested.  A  positive  relationship 
implies  that  as  one  variable  inereases  or  deereases,  the  other  variable  moves  in  the  same 
direetion.  A  negative  relationship  implies  that  as  one  variable  inereases  or  deereases,  the 
other  variable  moves  in  the  opposite  direetion.  It  is  useful  to  know  the  relationship  in  this 
researeh  to  apply  the  results  of  the  multiple  regression  model.  If  the  model  determines 
that  there  is  a  relationship  between  two  variables,  the  seattergram  will  suggest  the  nature 
of  that  relationship. 

Beeause  of  the  limited  data  available,  the  independent  variables  of  “Bundled 
Contraets”  and  “PBSCs”  will  not  be  plotted.  Three  data  points  are  not  enough  to  suggest 
a  trend  or  relationship.  Similarly,  beeause  the  variables  related  to  aequisition  reform  are 
binary,  they  eannot  be  plotted  in  the  seattergram  format.  Therefore,  the  independent 
variables  plotted  are:  Total  Contraet  Awards,  Total  Dollars  Awarded,  GWACs,  and 
MACS. 

Figures  3  through  10  on  the  following  pages  represent  eaeh  of  the  independent 
variables  relationship  with  the  two  dependent  variables  of  Total  Small  Businesses  and 
Total  New  Small  Businesses.  The  line  on  eaeh  table  represents  a  trend  in  the  relationship 
and  is  added  for  clarity.  Table  3  summarizes  each  of  the  possible  relationships  identified 
in  Figures  3  through  10. 
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Number  of  Small  Businesses 
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Figure  3,  Number  of  Small  Businesses  vs.  Total  Contract  Awards 
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Figure  4,  Number  of  New  Small  Businesses  vs.  Total  Contract  Awards 
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Number  of  New  Small  Businesses 


Total  Dollars  Awarded  (000s) 


Figure  5,  Number  of  Small  Businesses  vs.  Total  Dollars  Awarded 


Total  Dollars  Awarded  (000s) 


Figure  6,  Number  of  New  Small  Businesses  vs.  Total  Dollars  Awarded 
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Number  of  Small  Businesses 


Figure  7,  Number  of  Small  Businesses  vs.  Number  of  GWACs 


Figure  8,  Number  of  New  Small  Businesses  vs.  Number  of  GWACs 
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Number  of  Small  Businesses 


Figure  9,  Number  of  Small  Businesses  vs.  Number  of  MACs 


Figure  10,  Number  of  New  Small  Businesses  vs.  Number  of  MACs 
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The  results  of  several  of  the  scattergrams  are  more  clearly  evident  than  others,  but 
each  trend  line  reveals  a  possible  relationship.  Table  3  summarizes  those  relationships. 


Table  3.  Relationships  Between  Dependent  and  Independent  Variables 


Number  of  Small 

Businesses 

Number  of  New  Small 

Businesses 

Total  Contract  Awards 

Positive 

Slightly  Positive 

Total  Dollars  Awarded 

Positive 

Negative 

Number  ofGWACs 

Positive 

Negative 

Number  of  MACs 

Positive 

Positive 

Based  on  these  results,  one  could  assume,  for  example,  that  increasing  the  number  of 
MACs  awarded  will  increase  the  number  of  small  businesses  and  the  number  of  new 
small  businesses.  Such  an  assumption  at  this  point  would  be  flawed.  The  following  steps 
test  the  multiple  regression  model  to  see  whether  the  independent  variables  affect  the 
dependent  variables  or  not.  Without  the  knowledge  gained  through  the  multiple 
regression  model,  the  scattergram  results  have  limited  meaning. 


Step  4:  Testing  the  Complete  Multiple  Regression  Model 

As  mentioned  in  Step  1 ,  the  number  of  MACs  awarded  has  only  been  tracked  by  the 
Air  Force  since  FY  1996.  This  requires  an  adjustment  to  the  complete  model.  For  each 
dependent  variable,  the  model  will  be  tested  twice:  the  first  test  will  cover  the  entire  time 
period  but  not  include  the  MACs  independent  variable;  the  second  test  will  cover  the  time 
period  since  FY  1996  and  not  include  the  variables  related  to  acquisition  reform,  as  they 
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are  constant  through  the  covered  period.  This  section  provides  an  overview  of  the  results 
of  those  tests. 

Figure  1 1  provides  the  JMP  results  for  the  total  small  businesses  dependent  variable 
for  the  period  from  FY  1990  to  FY  2003.  The  independent  variables  included  are;  Total 
Contract  Awards,  Total  Dollars  Awarded,  GWACs,  FASA,  and  Clinger-Cohen. 


Summary  of  Fit 

RSquare 

0.883052 

RSquare  Adj 

0.80996 

Root  Mean  Square  Error 

275.4582 

Mean  of  Response 

4650.5 

Observations  (or  Sum  Wgts) 

14 

Analysis  of  Variance 

Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  5 

4583483.7 

916697 

12.0813 

Error  8 

607017.8 

75877 

Prob  >  F 

S.  Total  13 

5190501.5 

0.0015 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-609.8748 

933.9573 

-0.65 

0.5321 

Total  Contract  Awards 

0.2140573 

0.045641 

4.69 

0.0016 

Total  Dollars  Awarded 

-2.561  e-8 

3.506e-8 

-0.73 

0.4859 

SWACs 

-0.320371 

0.106998 

-2.99 

0.0172 

=ASA[0] 

-169.6682 

129.7906 

-1.31 

0.2274 

Slinger-Cohen[0] 

-83.61065 

128.5215 

-0.65 

0.5336 

Figure  11.  JMP  Results  for  Total  Small  Businesses  FY  1990  to  FY  2003 


The  least  squares  model  is: 

y-hat  =  -609,87  +  .21xi  -0,00000003x2  -  0,32x3  -  169,67x4  -  83,61x5  +  e 
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The  R-squared  score  reflects  that  88%  of  the  variation  is  accounted  for  within  the  model. 
To  be  useful,  the  F-Ratio  must  be  greater  than  the  F-Ratio  of  the  chosen  significance 
level.  Using  the  standard  F.os,  the  model  is  useful  if  the  F-Ratio  >  4.26.  For  this  model, 
the  F-Ratio  of  12.08  exceeds  the  F.05.  Individual  elements  are  subjected  to  the  t-test. 
There  are  two  components  to  the  t-test.  First,  the  individual  t-score  will  be  compared  to 
the  t.o5  score  and  if  it  is  larger  than  the  t.05  score,  it  is  an  indication  that  the  variable  is 
useful.  Second,  if  the  observed  significance  level  is  less  than  the  alpha  of  .05,  then  that 
also  is  an  indication  that  the  variable  is  useful.  The  t.05  score  for  this  portion  of  the  model 
with  thirteen  degrees  of  freedom  is  1.771.  Based  on  these  factors,  the  useful  variables  of 
this  model  are  Total  Contract  Awards  and  GWACs. 


Summary  of  Fit 

^Square 

0.983161 

^Square  Adj 

0.960709 

Root  Mean  Square  Error 

109.189 

Mean  of  Response 

4981.25 

Observations  (or  Sum  Wgts) 

8 

Analysis  of  Variance 

Eource  DF  Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  4  2088274.8 

522069 

43.7895 

Error  3  35766.7 

11922 

Prob  >  F 

0.  Total  7  2124041.5 

0.0054 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-338.6023 

533.1637 

-0.64 

0.5705 

Total  Contract  Awards 

0.2048412 

0.027327 

7.50 

0.0049 

Total  Dollars  Awarded 

-2.269e-8 

1 .705e-8 

-1.33 

0.2752 

SWACs 

-0.29755 

0.065402 

-4.55 

0.0199 

0.0300412 

0.027665 

1.09 

0.3570 

Figure  12,  JMP  Results  for  Total  Small  Businesses  FY  1996  FY  2003 
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Figure  12  provides  the  JMP  results  for  the  total  small  businesses  dependent  variable 
for  the  period  from  FY  1996  to  FY  2003.  The  independent  variables  ineluded  are:  Total 
Contraet  Awards,  Total  Dollars  Awarded,  GWACs,  and  MACs. 

The  least  squares  model  is: 

y-hat  =  -338,6  +  ,20xi  -0,00000002x2  -  0,30x3  +  0,03x4  +  e 

For  this  model,  the  R-squared  seore  jumps  to  98.3%  of  the  variation  explained.  The  F- 
Ratio  again  exeeeds  the  standard  F.05,  so  the  model  is  useful  for  predietion.  The  1 05  seore 
with  seven  degrees  of  freedom  is  1.895.  Based  on  this,  the  useful  variables  from  this 
model  are  Total  Contraets  Awarded  and  GWACs. 


Summary  of  Fit 

^Square  0.571106 

^Square  Adj  0.264754 

Root  Mean  Square  Error  635.208 

Mean  of  Response  2461 .923 

Observations  (or  Sum  Wgts)  1 3 

Analysis  of  Variance 


Source  DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  5 

3760948.2 

752190 

1 .8642 

Error  7 

2824424.7 

403489 

Prob  >  F 

0.  Total  12 

6585372.9 

0.2192 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-2806.797 

2172.286 

-1.29 

0.2373 

Total  Contract  Awards 

0.2947753 

0.105365 

2.80 

0.0266 

Total  Dollars  Awarded 

-6.253e-8 

8.115e-8 

-0.77 

0.4662 

SWACs 

-0.65701 

0.247008 

-2.66 

0.0325 

=ASA[0] 

321.85641 

305.6415 

1.05 

0.3273 

0linger-Cohen[0] 

-279.9835 

296.4038 

-0.94 

0.3763 

Figure  13,  JMP  Results  for  Total  New  Small  Businesses  FY  1991  to  FY  2003 
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Figure  13  provides  the  JMP  results  for  the  total  new  small  businesses  dependent 
variable  for  the  period  from  FY  1991  to  FY  2003.  The  independent  variables  ineluded 
are:  Total  Contraet  Awards,  Total  Dollars  Awarded,  GWACs,  FASA,  and  Clinger- 
Cohen.  The  least  squares  model  is: 

y-hat  =  -2806,8  +  .29xi  -0.00000006x2  -  0,66x3  -  321,86x4  -  279,98x5  +  e 

The  R-squared  seore  refleets  that  57%  of  the  variation  is  aeeounted  for  within  the  model. 
The  F-Ratio  does  not  exeeed  the  standard  F.os,  so  the  model  is  not  useful  for  predietion. 
The  t.05  seore  with  twelve  degrees  of  freedom  is  1.782.  Based  on  this,  the  useful 
variables  from  this  model  are  Total  Contraet  Awards  and  GWACs. 


Summary  of  Fit 

^Square 

0.807689 

^Square  Adj 

0.551274 

Root  Mean  Square  Error 

460.6142 

Mean  of  Response 

2360.875 

Observations  (or  Sum  Wgts) 

8 

Analysis  of  Variance 

Eource  DF  Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  4  2673224.7 

668306 

3.1499 

Error  3  636496.2 

212165 

Prob  >  F 

0.  Total  7  3309720.9 

0.1865 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-3335.463 

2249.152 

-1.48 

0.2347 

Total  Contract  Awards 

0.3239923 

0.115277 

2.81 

0.0673 

Total  Dollars  Awarded 

-4.309e-8 

7.191e-8 

-0.60 

0.5912 

SWACs 

-0.744442 

0.275897 

-2.70 

0.0739 

-0.09242 

0.116703 

-0.79 

0.4862 

Figure  14,  JMP  Results  for  Total  New  Small  Businesses  FY  1996  to  FY  2003 
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Figure  14  provides  the  JMP  results  for  the  total  new  small  businesses  dependent 
variable  for  the  period  from  FY  1996  to  FY  2003.  The  independent  variables  ineluded 
are:  Total  Contraet  Awards,  Total  Dollars  Awarded,  GWACs,  and  MACs.  The  least 
squares  model  is: 

y-hat  =  -3335,46  +  ,32xi  -0,00000004x2  -  0,74x3  -  0,09x4  +  e 

For  this  model,  the  R-squared  seore  jumps  to  80.7%  of  the  variation  explained.  The 
F-Ratio  again  fails  to  exeeed  the  standard  F.os,  so  the  model  as  a  whole  is  not  useful  for 
prediction.  The  t.os  score  with  seven  degrees  of  freedom  is  1.895.  Based  on  this,  the 
useful  variables  from  this  model  are  Total  Contract  Awards  and  GWACs.  However, 
none  of  the  variables  pass  the  observed  significance  level  alpha  test.  This  suggests  that 
none  of  the  independent  variables  are  useful  in  the  prediction  model. 

Table  4  summarizes  the  results  of  the  preliminary  whole  model  tests. 


Table  4,  Usefulness  of  Whole  Models  and  Independent  Variables 


Model 

Useful  as  a  Model? 

Useful  Independent  Variables 

Total  Small  Businesses 

FY  1990  to  FY  2003 

Yes 

Total  Contract  Awards 
GWACs 

Total  Small  Businesses 

FY  1996  to  FY  2003 

Yes 

Total  Contract  Awards 
GWACs 

Total  New  Small  Businesses 
FY  1991  to  FY  2003 

No 

Total  Contract  Awards 
GWACs 

Total  New  Small  Businesses 
FY  1996  to  FY  2003 

No 

None 

Step  5:  Testing  the  Reduced  Model 

The  new  model  will  be  run  using  only  the  independent  variables  deemed  useful  from 
the  complete  model.  The  only  independent  variables  used  in  each  model  will  be  Total 
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Contract  Awards  and  GWACs.  Figure  15  presents  the  results  from  the  reduced  total 
small  businesses  model.  Figure  16  presents  the  results  from  the  redueed  total  new  small 
businesses  model.  Eaeh  will  be  analyzed  using  the  same  tools  to  analyze  the  eomplete 
model. 


Summary  of  Fit 

^Square 

0.798506 

^Square  Adj 

0.761871 

Root  Mean  Square  Error 

308.3467 

Mean  of  Response 

4650.5 

Observations  (or  Sum  Wgts) 

14 

Analysis  of  Variance 

Eource  DF  Sum  of  Squares 

Mean  Square 

F  Ratio 

Model  2  4144646.8 

2072323 

21.7961 

Error  11  1045854.7 

95078 

Prob  >  F 

0.  Total  13  5190501.5 

0.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-978.763 

975.1368 

-1.00 

0.3371 

Total  Contract  Awards 

0.2017287 

0.04344 

4.64 

0.0007 

SWACs 

-0.24885 

0.106914 

-2.33 

0.0400 

Figure  15,  JMP  Results  for  Total  Small  Businesses  Reduced  Model 


The  least  squares  model  is: 

y-hat  =  -978,76  +  ,20xi  -0,25x2  +  e 

The  R-squared  score  is  79.9%.  The  F-Ratio  of  21.7961  does  exceed  the  standard  F.os, 
providing  evidenee  that  the  model  is  useful.  In  addition,  eaeh  independent  variable 
passes  the  t-test  of  usefulness. 
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Summary  of  Fit 

^Square  0.480248 

^Square  Adj  0.376298 

Root  Mean  Square  Error  585.0437 

Mean  of  Response  2461 .923 

Observations  (or  Sum  Wgts)  1 3 

Analysis  of  Variance 


Eource 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

2 

3162612.1 

1581306 

4.6200 

zrror 

10 

3422760.8 

342276 

Prob  >  F 

3.  Total 

12 

6585372.9 

0.0379 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

ntercept 

-1997.468 

1892.09 

-1.06 

0.3159 

Total  Contract  Awards 

0.2344635 

0.082932 

2.83 

0.0179 

SWACs 

-0.615523 

0.202882 

-3.03 

0.0126 

Figure  16,  JMP  Results  for  Total  New  Small  Businesses  Reduced  Model 


The  least  squares  model  is: 

y-hat  =  -1997,47  +  ,23xi  -0,62x2  +  e 

The  R-squared  score  is  48.0%.  The  F-Ratio  of  4.62  barely  exceeds  the  standard  F.os, 
providing  evidence  that  the  model  is  useful.  In  addition,  each  independent  variable 
passes  the  t-test  of  usefulness. 


Step  6:  Comparing  the  Reduced  Models  to  the  Complete  Models 

There  are  two  steps  to  comparing  the  usefulness  of  the  reduced  models  to  the 
complete  models.  First,  the  F-Ratios  will  be  compared.  A  higher  F-Ratio  reflects  a  more 
useful  model.  Second,  the  R-squared  scores  will  be  compared.  The  higher  the  R-squared 
score,  the  more  the  variance  that  is  accounted  for  within  the  model.  Table  5  compares  the 
F-Ratios  and  R-squared  scores  of  the  six  models  tested. 
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Table  5,  Comparison  of  Reduced  Models  to  Complete  Models 


Model 

F-Ratio 

R-squared  scores 

Complete  model — Total  Small 
Businesses — FY  1990  to  FY  2003 

12.08 

0.88 

Complete  model — Total  Small 
Businesses — FY  1996  to  FY  2003 

43.79 

0.98 

Complete  model — Total  New  Small 
Businesses —  FY  1991  to  FY  2003 

1.86 

0.57 

Complete  model — Total  New  Small 
Businesses —  FY  1996  to  FY  2003 

3.15 

0.808 

Reduced  model — Total  Small 
Businesses —  FY  1990  to  FY  2003 

21.8 

0.799 

Reduced  model — Total  New  Small 
Businesses —  FY  1991  to  FY  2003 

4.62 

0.48 

Based  on  the  idea  that  higher  F-Ratios  and  R-squared  scores  represent  more  effective 
models,  then  the  best  model  used  is  the  Complete  Model — Total  Small  Businesses — FY 
1996  to  FY  2003.  However,  it  is  important  to  note  that  the  Reduced  Model  covers  a 
longer  time  period.  Additionally,  although  the  Reduced  Model  may  appear  less  useful 
based  on  these  two  measures,  it  reflects  only  the  independent  variables  that  have  been 
determined  to  be  useful.  This  helps  isolate  the  importance  of  those  two  variables — the  R- 
squared  scores  of  the  Reduced  Models  represent  a  large  percentage  of  the  Complete 
Model,  with  a  reduced  number  of  variables  to  track. 


Step  7:  Testing  the  Hypotheses 

This  section  will  involve  testing  each  of  the  hypotheses  presented  in  Chapter  III  using 

the  results  of  both  the  Complete  Models  and  the  Reduced  Models. 

H^:  An  increase  in  Air  Force  contract  actions  will  increase  the  number  of 
Air  Force  small  business  contractors. 
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The  data  supports  HI.  Number  of  contract  actions  is  a  useful  independent  variable  in  the 

model.  In  addition,  the  scattergrams  confirm  the  positive  relationship  between  Air  Force 

contract  actions  and  the  number  of  Air  Force  small  business  contractors. 

H?:  An  increase  in  Air  Force  contract  actions  will  increase  the  number  of 
new  Air  Force  small  business  contractors. 

The  data  supports  H2.  Number  of  contract  actions  is  a  useful  independent  variable  in  the 

model.  In  addition,  the  scattergrams  confirm  the  slightly  positive  relationship  between 

Air  Force  contract  actions  and  the  number  of  new  Air  Force  small  business  contractors. 

H3:  An  increase  in  Air  Force  dollars  spent  will  increase  the  number  of  Air 
Force  small  business  contractors. 


H4:  An  increase  in  Air  Force  dollars  spent  will  increase  the  number  of  new 
Air  Force  small  business  contractors. 

The  data  do  not  support  H3  and  H4.  Air  Force  dollars  spent  is  not  a  useful  independent 
variable  in  either  model. 

H5:  An  increase  in  performance-based  service  contracts  will  decrease  the 
number  of  Air  Force  small  business  contractors. 


He:  An  increase  in  performance-based  service  contracts  will  decrease  the 
number  of  new  Air  Force  small  business  contractors. 


H?:  An  increase  in  bundled  contracts  will  decrease  the  number  of  Air 
Force  small  business  contractors. 


Hg:  An  increase  in  bundled  contracts  will  decrease  the  number  of  new  Air 
Force  small  business  contractors. 

Unfortunately,  the  lack  of  data  means  that  these  four  hypotheses  cannot  be  tested  using 
the  models  developed.  Although  the  variables  could  not  be  tested  in  the  model,  some 
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initial  findings  can  be  made  in  referenee  to  these  two  variables.  As  Table  1  shows,  the 
number  of  bundled  eontraets  has  deereased  eaeh  of  the  three  years  it  has  been  reeorded, 
from  55  in  FY  2001  to  20  in  FY  2003.  This  trend  is  not  surprising,  due  to  the  mandate  to 
eliminated  bundled  eontraets  and  the  speeial  proeedures  required  to  use  the  bundling 
teehnique.  PBSCs  have  inereased  in  eaeh  of  the  three  years,  from  2555  in  FY  2001  to 
6227  in  FY  2003.  Again,  with  the  mandates  requiring  that  50  pereent  of  all  serviee 
eontraets  be  performanee-based,  the  rising  number  should  be  expeeted. 

H9:  An  inerease  in  the  number  of  government  wide  ageney  eontraet 
aetions  will  deerease  the  number  of  Air  Foree  small  business 
eontraetors. 


Hio:  An  inerease  in  the  number  of  government  wide  ageney  eontraet 
aetions  will  deerease  the  number  of  new  Air  Foree  small  business 
eontraetors. 

The  number  of  government  wide  ageney  eontraets  does  appear  to  be  a  useful  independent 
variable  in  eaeh  model.  The  seattergrams  suggest  that  there  is  a  positive  relationship 
between  the  number  of  GWACs  and  the  total  number  of  small  businesses.  However,  the 
model  eonfirms  that  there  is  a  negative  relationship  between  these  two  variables  (negative 
estimate  value).  Based  on  this,  H9  and  HIO  are  not  rejeeted. 

Hn:  An  inerease  in  the  number  of  multiple  award  eontraet  aetions  will 
deerease  the  number  of  Air  Foree  small  business  eontraetors. 


H12:  An  inerease  in  the  number  of  multiple  award  eontraet  aetions  will 
deerease  the  number  of  new  Air  Foree  small  business  eontraetors. 


H13:  FASA  led  to  a  deerease  in  the  number  of  Air  Foree  small  business 
eontraetors. 
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Hi4:  FASA  led  to  a  decrease  in  the  number  of  new  Air  Force  small 
business  contractors. 

H^:  Clinger-Cohen  led  to  a  decrease  in  the  number  of  Air  Force  small 
business  contractors. 

H^:  Clinger-Cohen  led  to  a  decrease  in  the  number  of  new  Air  Force 
small  business  contractors. 

The  data  do  not  support  Hypotheses  1 1  through  16.  The  number  of  MACs,  FASA,  and 
Clinger-Cohen  are  not  useful  independent  variables  in  either  model. 

Testing  the  Proposed  Metrics  Using  the  Metrics  Construct 

Chapter  III  identified  five  questions  that  can  be  used  to  determine  whether  metrics  are 
effective.  Answering  the  five  questions  for  the  proposed  metrics  of  total  number  of  Air 
Force  small  business  contractors  and  total  number  of  new  Air  Force  small  business 
contractors  will  provide  the  analysis  needed  to  answer  the  second  research  question  in 
Chapter  V. 

The  first  requirement  is  that  the  measurement  be  quantifiable.  Both  proposed  metrics 
pass  this  test.  Not  only  can  they  be  quantified,  but  the  method  of  doing  so  is  clear.  The 
actual  count  can  be  obtained  through  an  analysis  of  the  JOOl  database,  as  performed  in 
this  research. 

The  second  question  requires  the  metrics  to  be  consistent  with  the  goals  of  the 
organization,  which  is  the  Air  Force  Small  Business  office  in  this  case.  Again,  both 
metrics  meet  this  requirement.  Both  measures  fit  well  with  the  outlined  strategies  and 


83 


goals  included  in  the  Air  Force  Small  Business  and  HBCU/MI  Program  Plan  for  Fiscal 
Years  2003-2007. 

The  third  question  requires  that  the  organization  be  able  to  establish  speeifie,  research- 
based  targets  based  on  the  metries.  The  real  issue  here  is  whether  the  targets  are  based  on 
research  or  merely  arbitrary  numbers.  This  research  provides  historieal  figures  for  eaeh 
of  the  metries  for  over  a  deeade.  Based  on  these  numbers,  speeifie  targets  ean  be  set  for 
eaeh  metric.  Each  metric  meets  the  requirements  of  the  third  question. 

The  fourth  question  requires  that  a  strategy  to  reaeh  the  targets  of  the  metrie  must  be 
able  to  be  developed.  This  researeh  has  identified  several  faetors  that  affect  the  numbers 
for  eaeh  metrie.  Other  faetors  exist  that  still  need  to  be  determined.  By  understanding 
how  these  faetors  relate  to  the  metries,  the  Air  Force  Small  Business  office  will  be  able  to 
develop  strategies  to  reaeh  the  targets  set  for  eaeh  metrie.  Again,  eaeh  metric  passes  this 
test. 

The  final  question  requires  the  metrie  to  have  the  support  of  organizational  leadership. 
This  researeh  cannot  determine  the  answer  to  this  question.  The  leadership  of  the  Air 
Foree  Small  Business  office  must  determine  that  these  metries  are  useful  and  important 
and  provide  their  full  support  of  them  in  order  for  either  metrie  to  meet  this  requirement 
and  to  be  effectively  used  by  the  organization. 

Conclusion 

In  a  perfeet  world,  the  ehosen  independent  variables  would  fully  explain  the  varianee 
in  the  dependent  variables.  Additionally,  one  or  two  variables  would  be  identified  that 
would,  by  themselves,  allow  predietions  to  be  made  regarding  the  dependent  variable. 
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Such  has  not  been  the  ease  with  these  models  and  variables.  Nonetheless,  the  models  are 
statistieally  useful  for  predieting  the  dependent  variables.  Two  independent  variables 
have  been  identified  as  being  important  eontributors  to  the  variation  in  the  dependent 
variables.  By  following  the  steps  outlined  in  the  methodology  of  Chapter  III,  the  models 
have  allowed  the  hypotheses  to  be  tested  despite  limited  data.  These  hypotheses  tests 
provide  information  to  help  answer  the  main  questions  of  this  researeh.  Those  questions 
will  be  addressed  in  Chapter  V. 
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Chapter  V:  Conclusions  and  Recommendations 


Chapter  V  uses  the  information  obtained  through  the  literature  review  of  Chapter  II 
eombined  with  the  data  analysis  of  Chapter  IV  to  answer  the  researeh  questions  of 
Chapter  I.  It  then  looks  at  the  contributions  of  the  research,  the  limitations  of  the 
research,  and  the  recommendations  of  the  researcher  for  future  actions  and  research. 

The  Research  Questions 

This  research  began  by  presenting  two  main  research  questions  in  the  opening  chapter. 
They  are: 

1)  Have  acquisition  reform  initiatives  negatively  impacted  small  business 
participation  in  Air  Force  contracting? 

2)  Should  the  Air  Force  Small  Business  office  incorporate  the  measures  of 
total  small  business  contractors  and  total  new  small  business  contractors 
into  their  metrics? 

The  literature  review  of  Chapter  II,  the  methodology  of  Chapter  III,  and  the  analysis 
of  Chapter  IV  provide  answers  to  these  questions. 

The  acquisition  reforms  of  the  1990s  appear  to  have  had  little,  if  any,  impact  on  small 
business  participation.  The  number  of  small  businesses  and  number  of  new  small 
businesses  participating  in  Air  Force  contracting  has  been  relatively  stable  or  even 
increasing  over  the  past  thirteen  years  (see  Figure  1).  Of  the  independent  variables  tested 
in  the  model  related  to  acquisition  reforms,  only  the  number  of  government  wide  agency 
contracts  has  been  determined  to  affect  the  participation  of  small  businesses.  The  answer 
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to  the  question  of  whether  aequisition  reform  has  negatively  impacted  small  business 
participation  must  be  answered  negatively  based  on  the  factors  considered  within  this 
research. 

The  second  question  can  be  answered  based  on  the  construct  developed  in  Chapter  III 
and  tested  in  Chapter  IV.  Both  metrics  are  viable  candidates  for  use  by  the  Air  Force 
Small  Business  office.  Leadership  of  that  office  must  provide  full  support  for  the  metric, 
however,  including  developing  a  strategy  to  achieve  increased  small  business 
participation  and  linking  it  to  the  overall  Small  Business  office  strategy  will  ensure  that 
these  additional  metrics  are  effective.  This  research  concludes  that  the  Air  Force  Small 
Business  office  should  use  both  metrics  as  performance  measurements. 

Contributions  of  the  Research 

This  effort  makes  two  related  contributions  to  small  business  research.  Significantly, 
the  research  has  found  that  the  Air  Force  has  been  able  to  maintain  and  even  increase  the 
number  of  small  businesses  with  contracts.  This  goes  directly  against  findings  for  the 
federal  government  as  a  whole  and  therefore  what  may  have  been  anticipated  for  the  Air 
Force.  The  reasons  for  this  difference  may  not  ultimately  be  clear,  but  the  data 
demonstrates  that  the  Air  Force  has  not  experienced  the  same  reductions  in  small 
business  contractors  that  the  federal  government  as  a  whole  has  experienced  according  to 
research  performed  by  the  Office  of  Management  and  Budget. 

The  second  contribution  of  the  research  is  that  acquisition  reform  has  had  limited 
impact  on  small  business  participation.  Despite  expectations  or  fears  that  some  actions 
have  negatively  impacted  small  businesses,  this  research  concludes  that  they  have  not. 
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The  important  point  is  that  with  the  iterative  nature  of  acquisition  reform,  the  expected 
impact  of  individual  reforms  may  not  be  what  is  anticipated.  Unfortunately,  this  research 
is  unable  to  determine  why  reforms  that  on  the  surface  appear  to  be  harmful  to  small 
business  participation  have  not  had  a  negative  effect. 

Limitations  to  the  Research 

The  primary  limitation  to  this  research  is  the  data.  Not  only  does  the  availability  of 
data  constrain  the  independent  variables  that  could  be  included  in  the  model,  but  two 
variables  in  the  original  model  could  not  be  tested  due  to  limited  data  points.  In  addition 
to  the  limited  nature  of  the  data,  the  accuracy  of  the  data  must  also  be  questioned.  David 
E.  Cooper,  Director  of  Contracting  Issues  for  the  General  Accounting  Office,  testified 
before  Congress  in  2003  on  flaws  in  the  Federal  Procurement  Data  System  (FPDS). 
According  to  Cooper,  of  the  49,366  companies  who  were  coded  as  small  businesses  in 
FPDS  and  received  contract  awards  in  fiscal  year  2001,  5,341  of  those  companies  also 
received  awards  coded  as  large  businesses.  This  suggests  that  many  of  the  companies  in 
the  database  are  potentially  miscoded  in  regards  to  business  size  (Cooper,  2003). 
Congresswoman  Velazquez  believes  that  this  miscalculated  data  means  that  “the  state  of 
contracting  opportunities  for  small  businesses  is  much  worse  than  was  previously 
believed”  (Velazquez,  2003:3).  It  is  important  to  note,  however,  that  miscoded  data  is 
not  always  going  to  have  a  negative  impact  on  the  final  results.  It  is  just  as  possible  that  a 
business  coded  as  a  large  business,  and  therefore  not  counted  towards  small  business 
goals,  could  be  a  small  business.  The  important  point  is  that  inaccurate  and  incomplete 
data  can  greatly  affect  the  results  of  this  research. 
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A  second  limitation  of  the  research  is  the  regression  model  used.  Because  the  results 
can  only  be  applied  to  the  data  used  in  the  model,  it  lacks  predictive  power  to  suggest 
how  small  business  participation  might  be  affected  in  the  future.  The  model  is  able  to 
meet  the  needs  of  the  research  design  in  identifying  relationships,  but  a  predictive  ability 
could  enhance  the  research. 

Recommendations 

The  following  recommendations  can  be  sorted  into  two  groups:  recommendations  for 
policymakers  in  the  small  business  office  and  recommendations  for  future  research.  Each 
recommendation  is  based  on  results  of  the  research  and  observations  of  the  researcher. 

Measuring  the  number  of  small  businesses  contracting  with  the  Air  Force  and  the 
number  of  new  small  businesses  that  enter  that  group  should  be  metrics  used  by  the  Air 
Force  Small  Business  Office.  The  metrics  must  be  linked  to  strategic  goals  that  will  lead 
to  the  development  of  strategic  actions  and  plans.  The  Air  Force  Small  Business  Office 
needs  to  develop  strategies  that  support  and  encourage  current  and  potential  small 
businesses  to  compete  for  Air  Force  contracts. 

Although  acquisition  reforms  covered  in  this  research  have  not  negatively  impacted 
small  business  participation,  there  is  no  guarantee  that  current  and  future  reforms  will 
not.  Currently,  the  Air  Force  Small  Business  Office  is  closely  involved  in  the  protection 
of  small  business  participation  as  reforms  are  implemented.  These  actions  need  to  be 
maintained.  Research  such  as  this  should  be  an  iterative  process  to  see  how  any  reforms 
are  impacting  small  business  participation. 
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Opportunities  for  future  researeh  exist.  The  model  developed  here  could  be  used  to 
incorporate  other  independent  variables,  such  as  those  used  by  Mayer  (1992),  to  identify 
the  effects  of  the  budget,  economic,  and  political  cycles  on  small  business  participation. 
Research  that  attempts  to  understand  why  individual  small  businesses  choose  or  choose 
not  to  contract  with  the  Air  Force  can  provide  an  alternative  perspective  that  can  guide 
policy  formation.  Research  that  addresses  the  quality  of  the  relationships  between  small 
businesses  and  the  Air  Force  is  also  lacking.  Qualifying  relationships  as  successful  or 
unsuccessful  and  determining  what  factors  influence  each  outcome  can  help  focus  Air 
Force  outreach  efforts.  Finally,  research  that  addresses  Air  Force  small  business  outreach 
is  also  needed.  What  is  the  success  of  current  efforts  and  how  can  this  tool  be  used  to 
increase  the  number  of  new  small  businesses  doing  business  with  the  Air  Force?  Any  of 
these  topics  are  appropriate  for  further  exploration  and  research  in  attempting  to 
understand  the  participation  of  small  businesses  in  Air  Force  contracting. 
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